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ABSTRACT 



This monograph, part of a series on student assessment, 
reports on the results of a national survey that examined institutional 
support for undergraduate student assessment. The study provides a national 
profile of student assessment initiatives at associate of arts institutions 
and compares it to assessment practices and support patterns found in all 
types of postsecondary institutions. It also offers advice for 
administrators, faculty, and staff in associate of arts institutions who are 
designing student assessment approaches and support processes that are likely 
to foster institutional improvement. Also included is the survey instrument 
used to gather study data, which institutions can use to examine patterns for 
student assessment on their own campuses. The monograph is organized in nine 
sections; they are: (1) "Introduction and Overview," (2) "Perspectives on 

Student Assessment in Higher Education," (3) "The Influence of External 
Groups," (4) "Approaches to Student Assessment," (5) "Organizational and 
Administrative Support Patterns," (6) "Assessment Management Practices and 
Policies," (7) "Uses and Impacts of Student Assessment," (8) "Key 
Relationships," and (9) "Executive Summary: Research Results and 
Recommendations." Appended is the survey instrument, the "Inventory of 
Institutional Support for Student Assessment for Associate of Arts 
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1. INTRODUCTION AND OVERVIEW 



This monograph reports the results of a national research survey that 
examines institutional support for undergraduate student assessment. It 
provides a national profde of student assessment initiatives at associate of arts 
institutions; useful advice for administrators, faculty, and staff who are 
designing student assessment approaches and support processes that are likely 
to foster institutional improvement; and an instrument that institutions can use 
to examine patterns for student assessment on their own campuses. 

This report is a result of the National Center for Postsecondary Improvement (NCPI) 
research program examining organizational and administrative support for student assessment. In 
1997-98, NCPI researchers conducted a major literature review followed by a national survey on 
institutional support for student assessment. While other dissemination efforts have focused on a 
research audience, this report is specifically written for administrators, faculty, and staff in 
associate of arts institutions who are involved with student assessment on their campuses. 

Purpose of the Monograph 

Administrators and faculty leaders have little credible and verifiable evidence to guide their 
planning and decision-making regarding student assessment. The literature on student assessment 
offers many descriptions of student assessment practices at a variety of postsecondary institutions 
(Banta & Associates, 1993; Banta, Lund, Black, & Oblander, 1996) and a number of limited 
surveys have collected information concerning student assessment measures and methods used by 
focused groups of institutions (Cowart, 1990; El-Khawas, 1988, 1989, 1990, 1991, 1992, 1993, 
1995, 1996; Johnson, Prus, Andersen, & El-Khawas, 1991). There has been comparatively less 
consideration of how colleges and universities can develop an effective institutional strategy for 
assessment or of organizational and administrative practices that support internal engagement in 
assessment and enhance the use of student assessment data. Guidelines for planning and 
implementing student assessment efforts are available (American Association for Higher Education 
[AAHE], 1992; Ewell, 1988a, 1988b; Rossman & El-Khawas, 1987) but largely missing is any 
systematic examination of the relationships among various ways institutions have approached and 
supported student assessment and the likelihood of their reaping positive impacts from student 
assessment efforts (Banta et al., 1996; Ewell, 1988b; Gray & Banta, 1997). Furthermore, there 
has been little research focused specifically on student assessment within associate of arts 
institutions. 

Understanding the national profile of undergraduate assessment activities and support 
patterns found in associate of arts institutions is important for several reasons. First, it is helpful 
for institutions to know what their peers are doing in relation to student assessment. How are 
other institutions responding to external demands for student assessment? What approaches to 
student assessment are they using? How are they supporting and promoting student assessment 
through their practices and policies? How are they using student assessment data? What impacts 
has student assessment had on faculty as well as institutional and student performance? Finally, if 
student assessment activities do make a difference in terms of improved teaching and learning on 
campuses, how does that happen? What external influences, institutional approaches, and 
organizational and administrative support practices encourage internal involvement-in student 
assessment and positive impacts from student assessment data? 

This monograph will answer these questions through three primary objectives. First, it 
will provide a national profile of current student assessment practices and institutional support 
patterns within associate of arts institutions and compare this to assessment practices and support 
patterns found in all types of postsecondary institutions. A second major objective is to provide 
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advice for administrators, faculty, and staff in associate of arts institutions on designing student 
assessment approaches and support processes that are likely to foster institutional improvement. 
Finally, this report includes the survey instrument used to gather data for this study in Appendix A. 
Institutions can use this inventory to examine student assessment patterns on their own campuses. 

We will discuss the student assessment methods used by institutions in this report, but it is 
not our intent to examine the specific instruments used or the measurement issues associated with 
their use. For a detailed literature review on student assessment, please consult Improving 
Organizational and Administrative Support for Student Assessment: A Review of the Research 
L iterature (Peterson, Einarson, Trice, & Nichols, 1997). The data from our survey research have 
been summarized in several forms including numerous conference papers and journal articles, and 
a technical report entitled Institutional Support for Student Assessment: Methodology and Results 
fit a National Survey (Peterson, Einarson, Augustine, & Vaughan, 1999). Readers who are 
interested in the statistical analyses conducted on the student assessment data should consult the 
technical report. While this report relies on those statistical analyses, it will not provide statistical 
details. 

Definition of Student Assessment 

Our focus in this report is on undergraduate student assessment from an institutional 
perspective. We are interested in what approach institutions use to assess student performance, 
how institutions are organized to promote and support student assessment, and how they use 
student assessment to improve student, faculty, and academic performance in the institution. 

In this monograph, we use the term student assessment to refer to activities other than 
traditional end-of-course grading that are used to measure a diverse array of dimensions of student 
performance or development. Institutions may decide to engage in student assessment for a variety 
of reasons or differing purposes. Cognitive, affective, and behavioral dimensions of student 
performance and development may be assessed. Assessment efforts may be directed toward 
students as they enter the institution, during their enrollment, or after they exit from the institution. 
Student assessment may use a variety of methods and may be planned and executed centrally 
within institutions or in a decentralized manner through the actions of individual academic units. 
This report will address all these dimensions of student assessment. 

Preview of the Monograph 

This report is presented in nine sections. Following this introduction, the second section, 
perspectives on student assessment, provides a brief recent history of the student assessment 
movement in general and in associate of arts institutions specifically. The conceptual framework of 
institutional support for student assessment that was developed in the literature review (Peterson et 
al., 1997) and guided the development of the survey instrument is then presented. Section two 
ends with a brief summary of the methods used to conduct the research on institutional support for 
student assessment. 

Sections three through seven present the results of a national survey on institutional support 
for student assessment. In each of these sections, information is presented for both associate of 
arts institutions and all institutions that responded to the study. Section three focuses on the 
influence of external groups, such as state agencies and accreditors, on institutional student 
assessment patterns. In this section, the influence of states on institutional student assessment 
activity is considered for public institutions only. Section four focuses on institutions’ approaches 
to student assessment. This section is followed by a summary of the institution- wide support 
patterns evident in the study. Section six discusses assessment management practices and policies 
used by institutions to promote or assure the use of student assessment information. Section seven 
reports on the institutional uses and impacts of student assessment. 
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Section eight is based on the statistical analyses conducted on the national survey data. It 
presents the findings from analyses of three sets of key relationships examined in the study. The 
first focuses on the influence of external groups on institutional approaches to student assessment. 
The second examines the relationship of external influences, institutional context, assessment 
approaches, institution-wide support patterns, and assessment management practices and policies 
to the institutional uses of student assessment data. The third analysis examines the relationship of 
external influences, institutional context, assessment approaches, institution- wide support patterns, 
and assessment management practices and policies to the impacts of engaging in student 
assessment. Section nine presents a summary of our research findings and related 
recommendations for specific assessment activities within associate of arts institutions. It 
concludes with general guidelines for institutions to use to examine and redesign or plan their 
student assessment process and functions. 
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2. PERSPECTIVES ON STUDENT ASSESSMENT IN HIGHER EDUCATION 

An overview of the development of student assessment in higher education 
and in associate of arts institutions suggests the need for the institutional 
perspective that guides this report. A guiding framework conceptualizes 
seven domains of an institution’s student assessment strategy: institutional 
context, external influences, approach adopted, institutional support patterns, 
assessment management practices and policies, assessment culture and 
climate, and institutional uses and impacts of student assessment. 

This monograph concurs with the perspective of other scholars of student assessment. We 
agree that, at its best, student assessment is not simply an exercise to gather data or respond to 
accountability requirements. Rather, an effective student assessment approach is one that gathers 
information about selected aspects of students' characteristics, achievements, and experiences and 
uses this information to shape institutional policies, processes, and practices in ways that lead to 
improved student performance and institutional functioning (AAHE, 1992; Banta & Associates, 
1993; Ewell, 1984, 1987c, 1988b; Jacobi, Astin & Ayala, 1987). 

Growth of Student Assessment 

The assessment of student performance in higher and postsecondary education is not a new 
concept or phenomenon. The first College Board examination designed to assess student learning 
outcomes on a national scale was administered in 1901. The ensuing decades are marked by 
events that reflect a growing concern with assessing college student performance, such as: the 
emergence of a regional accreditation focus on student assessment; the establishment of university- 
based and national testing centers; and the development of broadened taxonomies of student 
outcomes (Resnick & Goulden, 1987; Sims, 1992). However, these earlier developments pale in 
comparison to the emergence in the mid 1980s of student assessment as an important focus of 
educational policy at the national, state, and institutional level — a focus that continues today. 

This heightened interest in student assessment is the result of many broad forces. In the 
larger societal and political arena, concerns about consumer protection, the rising costs of 
education, the training and human resource needs for state and regional economic development, 
and fiscal pressures on state and federal government all have shaped public interest in the 
educational contribution of higher education institutions. At the institutional level, the challenges 
of expansion in the 1950s and 60s, enrollment and financial constraints in the 1970s, and new 
educational demands in the 1980s have shifted the managerial focus of performance by higher 
education institutions from resource adequacy, to efficiency, to effectiveness, to broader concerns 
for academic and institutional quality. Consequently, over the past decade and a half the 
assessment and improvement of student performance has been the focus of much of this discussion 
and of many efforts both external to and within colleges and universities. 

A variety of specific activities at the national level — reports, guidelines, legislation, 
educational goals, and funding — have been credited with providing the initial stimulus for the 
student assessment movement in higher education in the last decade and a half. A series of national 
reports that critically examined the quality of education were published in the mid 1980s. The first, 
A Nation at Risk: The Imperative for Educational Reform (National Commission on Excellence in 
Education, 1983), prompted calls for reform in elementary and secondary education. The 
following year Involvement in Learning: Realizing the Potential of American Higher Education 
(National Institute of Education, 1984) addressed the conditions required for improving the quality 
of undergraduate education. Institutions were given three recommendations: set high expectations 
for student learning, actively involve students in learning, and develop an institutionalized process 
for assessing student learning. Subsequent reports such as To Reclaim a Legacy: A Report on the 
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Humanities in Higher Education (Bennett. 1984) and Integrity in the College Curriculum: A 
Report to the Academic Community (Association of American Colleges, 1985) continued this 
focus on issues of undergraduate education quality and assessment. In 1988 the U.S. Department 
of Education revised its Criteria for Recognition of Accrediting Agencies, stipulating that agencies 
must require educational institutions and programs to (1) clearly specify their educational objectives 
and (2) conduct student assessment to determine whether they are achieving these standards. In 
addition, the enactment of legislation such as the "Student Right to Know and Campus Security 
Act" and "Ability to Benefit" legislation (Education Commission of the States, 1991), adoption of 
the National Education Goals (Lenth, 1993, 1996; Nettles, 1995), and provision of federal funds 
for the development of institutions' assessment projects (Cook, 1989) contributed to the early 
momentum of the assessment movement. 

State governments, reflecting these national developments, have also played a key role in 
stimulating postsecondary student assessment. The first state-level mandate for student assessment 
appeared when the Florida state legislature directed the higher education system to develop the 
College-Level Academic Skills Test in 1982. Since then, the number of states involved in student 
assessment has increased steadily with all but four of fifty states now reporting some type of 
student assessment initiative (Cole, Nettles, & Sharp, 1997). States have varied greatly in their 
approaches to student assessment policy, but there is evidence of a general shift in state-level 
approaches over the past decade. From the mid to late 1980s state assessment initiatives generally 
emphasized institutional improvement as their primary purpose and largely permitted institutions to 
design their own assessment efforts. Since 1990, states have placed greater emphasis on student 
assessment as a means of responding to external demands for accountability and have increasingly 
elected to mandate the content and form of institutions' assessment approaches. 

Regional accreditation associations also have emerged as an important influence on student 
assessment. Regional accrediting associations vary in the length of time they have been involved 
in student assessment, their specific reporting requirements, and the range of assessment-related 
services provided to member institutions (Cole et al., 1997). Since the federal government revised 
its criteria for recognizing accrediting agencies in 1988, all six regional accreditation associations 
have required member institutions to undertake and document some form of student assessment 
activity. In tum, a growing number of institutions have reported that accreditation requirements are 
an important reason for deciding to engage in and increase their student assessment efforts (El- 
Khawas, 1995; Johnson, et. al., 1991). 

Since the mid 1980s the number of postsecondary institutions engaged in some form of 
student assessment activity has steadily increased (El-Khawas, 1988, 1990, 1995). However, by 
1990 only a small proportion of institutions have embarked on comprehensive student assessment 
programs (El-Khawas, 1990; Hexter & Lippincott, 1990; Johnson et al., 1991). Fewer still have 
reported achieving observable impacts from their student assessment efforts (Astin, 1991; 

Hutchings & Marchese, 1990). So it seems that many colleges and universities are investing 
faculty and administrative time and effort in student assessment, but few are benefiting from its 
potential to improve student and institutional performance. 

Characteristics of Associate of Arts Institutions 

Institutional type is an important dimension to consider when examining the approach an 
institution takes toward student assessment. To date, there has been little examination of 
differences in effective organizational and administrative practices for promoting student 
assessment within specific types of institutions. Yet, effective strategies and practices for 
planning, promoting, and implementing undergraduate student assessment vary according to the 
type of postsecondary institution being considered — for example, what works in promoting 
student assessment in associate of arts institutions will be different from what works in research 
institutions. In our national study of institutional support for student assessment, we found that 
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there were many statistically significant differences by institutional type on the approaches to, 
support for, practices and policies regarding, and uses and impacts of student assessment reported 
by institutions, and that they were typically stronger than differences by other institutional 
characteristics such as size or whether the institution is private or public. 

The challenges associate of arts institutions face as they develop student assessment 
programs are quite different from those faced by other institutions. These differences stem, in 
part, from the institutional mission, curricular focus, governance structure, faculty roles and 
responsibilities, and the student clientele typically associated with associate of arts colleges. 

The missions of associate of arts institutions are usually quite broad, encompassing such 
diverse goals as occupational training, general education preparation, and welfare to work 
transitions. The curriculum found at associate of arts institutions is also necessarily broad to meet 
the institutional mission. While many associate of arts institutions have broad curriculum ranging 
from occupational to traditional, often the traditional curriculum will aggregate specialized 
disciplines from the humanities and liberal arts and science areas into a general “transfer program” 
for those students who plan to transfer. This aggregation may make it difficult to assess students 
in a specific liberal arts “program” as these students are allowed to take quite a number of different 
courses to fulfill their program requirements. 

Associate of arts institutions also often have somewhat unique governance structures. 
Administrators at these institutions typically wield more power than their peers at four-year 
institutions. Compared to faculty at other institutional types, associate of arts faculty generally 
have less power and autonomy (Cohen & Brawer, 1996; Frye, 1994; Reyes & Twombly, 1987; 
Seidman, 1985). This lack of autonomy suggests that administrators may have more influence 
over decisions to assess students than do administrators in four-year colleges and universities. 
Conversely, the lack of autonomy may mean that faculty at associate of arts institutions who want 
to explore assessing their students will not have the freedom or the support necessary to take 
advantage of professional development opportunities on student assessment. 

Associate of arts faculty are also different from faculty at four-year colleges and 
universities. Some researchers believe that because associate of arts faculty are more oriented 
toward teaching (Jemmott, 1992/1993), they will be more receptive to student assessment 
techniques that will lead to improving the teaching and learning process. Other researchers believe 
that associate of arts faculty may be less willing to engage in student assessment, as they typically 
have more limited research experience and expertise than do their four-year college peers (Palmer, 
1993). These researchers stress that familiarity with conducting research is important in 
developing both comfort and skill in assessing students. 

The characteristics of the student bodies at associate of arts institutions are also different 
from those at other institutions. Students may be interested in completing an associate degree 
program, transferring to a four-year institution, or meeting a more immediate goal such as 
occupational retraining. The higher percentage of part-time students at associate of arts institutions 
means that many are on campus for short time periods, have lower retention rates, and take longer 
to finish their programs. It may be difficult to involve these students in assessment activities 
outside of the classroom. Furthermore, Seybert (1994) cited the diversity of student background, 
employment status, and preparation as issues associate of arts institutions must address as they try 
to develop a comprehensive assessment plan. 

Palmer (1993) noted that many two-year institutions collect and analyze student data, but 
stop short of trying to apply standards to the analysis. In these cases, the decision seems to have 
been made that there are simply too many student goals to take into account. Applying any kind of 
general standards to the outcomes would not be reflective of the varying realities. Kreider and 
Walleri (1988) argued that two-year colleges should focus instead on a student-success approach 
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that is tied to stated student intentions. It is not clear from the literature how frequently this practice 
is used. However, extant research shows that entry-only assessment is more common than entry 
and exit assessment (Alfred, Peterson, & White 1992; Cowart, 1990). 

Despite the potential difficulties of assessing diverse students in associate of arts 
institutions, these institutions have been cited as developing quite advanced information systems 
for student tracking (Walleri & Seybert, 1993). These systems are sometimes developed in 
collaboration with other two- and four-year institutions located in the same state to allow for 
tracking students who transfer between and among institutions. Such systems are extremely 
helpful in assessing students. Although the existing literature includes references to these 
information systems and some broader institutional case studies of student assessment, it does not 
describe the current assessment activities conducted throughout the country by associate of arts 
institutions. 



A Framework of Institutional Support for Student Assessment 

An institution’s organizational and administrative context can be a powerful source of 
influence on administrators', faculty members' and students' behaviors and perceptions, shaping 
their views of what the institution values and which activities are important to engage in. Colleges 
and universities purposefully create a variety of structures, processes, practices, and policies to 
support and implement student assessment. Institutions have varied greatly in their approaches to 
assessing student performance and the ways in which they have organized to support their 
assessment efforts, a finding that is hardly surprising given the great diversity and relative 
autonomy of colleges and universities in the American postsecondary system. Yet there is little 
systematic evidence available concerning the specific manner in which postsecondary institutions 
have organized to support student assessment and the differential results of these efforts. For these 
reasons, we have chosen to examine student assessment from a research-based and institutional 
perspective. 

An extensive literature review identified what is currently known about the organizational 
and administrative context for student assessment in postsecondary institutions. A detailed 
description of the literature review process and findings is available in other publications (Peterson 
et al., 1999; Peterson et ah, 1997). From this review, we developed a framework of institutional 
support for student assessment. This framework, displayed in Figure 1, consists of seven 
interacting domains. It conceptualizes how institutions respond to external pressures for student 
assessment, how they approach student assessment, the organizational and administrative patterns 
they use to support student assessment, what assessment practices and policies they have adopted 
to promote student assessment, their culture and climate for student assessment, and how student 
assessment information is used by and has impacts on institutions. A seventh domain, institutional 
context, reflects the fact that these patterns probably differ significantly by institutional type, size, 
and whether the institution is private or public. The culture and climate domain is not examined in 
this report and is the focus of future research activity. The other domains are briefly described 
below. A complete listing of the variables included in each domain is provided in Appendix B. 

External Influences on Student Assessment . A variety of external constituencies have 
played an important role in initiating and shaping student assessment efforts within postsecondary 
institutions. In particular, state-level initiatives and regional accreditation associations may exert 
strong direct influences through their requirements for student assessment. The business 
community, private foundations, and professional higher education associations may serve as 
weaker sources of influence — whether as participants in institutions' assessment efforts, as 
consultants or sources of resource materials, or as providers of funds for assessment-related 
activities. National efforts appear to have largely played an indirect role in influencing institutions' 
assessment efforts. 
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Figure 1. Framework of Institutional Support for Student Assessment 
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Institutional Approach to Student Assessment. Institutional approach to student assessment 
refers to institutions’ decisions regarding the collection and analysis of student assessment 
information. Important dimensions along which student assessment approaches can be 
differentiated include: the type or content of student assessment measures; the extent or array of 
student assessment data collected; the instruments and methods used to collect student assessment 
data; and the analyses conducted and reported for collected data. 

Organizational and Admin istrative Support for Student Assessment . The literature on 
student assessment identifies five important dimensions of organizational or administrative 
behavior patterns that may support student assessment efforts and enhance the likelihood that 
assessment will contribute to improvements in institutional performance: the institution-wide 
support strategy for student assessment; institution-wide administrative and governance activities 
used to promote student assessment; leadership and faculty support for student assessment; 
planning and coordination of student assessment; and procedures used to evaluate and revise 
student assessment efforts. 

Assessment Manag ement Practices and Policies . Assessment management practices and 
policies refer to specific practices, policies, procedures, or activities intentionally devised by 
institutions to implement and support their student assessment efforts. A number of content 
activity areas of this domain have been identified including: academic resource allocation; student 
information systems; internal access to student information; distribution of assessment reports and 
studies; student involvement in assessment; professional development for student assessment; 
training for student affairs staff; faculty evaluation and rewards; and academic planning and review 
processes. 

Institutional Uses and Impa cts of Student Assessment . Institutions can use the information 
collected through student assessment processes to contribute to improvement in institutional and 
student performance. Three broad domains of institutional uses and impacts of student assessment 
information are suggested in the literature. These are the use of assessment information for 
institutional decision making, its impact on various internal constituents and phenomena, and its 
impact on the institution’s relationships with its external constituents. 

Institutions may use assessment information within several areas of academic decision 
making such as academic planning and review, academic mission and goals, resource allocation, 
and faculty evaluation and rewards. Student assessment information may have both internal 
impacts, such as stimulating faculty interest in teaching and enhancing student learning, and 
external impacts, such as influencing state funding or re-accreditation decisions. The primary 
concern of our framework is to examine the relationship of external influences, institutions’ 
assessment approaches, patterns of organizational and administrative support for assessment, and 
assessment management practices and policies to institutional uses and impacts of assessment 
information. 

Institutional Context. Broad institutional characteristics such as institutional type, size, and 
whether the institution is public or private are expected to moderate external influences on 
assessment, the institutional approach to student assessment, organizational and administrative 
support patterns, assessment management practices and policies, and institutional uses and impacts 
of assessment information. 

Description of National Survey 

Based on our review of the student assessment literature, we developed a survey 
instrument or inventory called "Institutional Support for Student Assessment" (ISSA). The 
instrument focuses on the assessment of undergraduate students conducted by postsecondary 
institutions — not individual faculty or academic sub-units within institutions. The instrument is 
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designed as a comprehensive inventory of external influences on institutions' student assessment 
efforts, institutional approaches to student assessment, organizational and administrative support 
patterns for student assessment, assessment management practices and policies used, and 
institutional uses and impacts of student assessment information. The instrument was intended to 
assist institutions in obtaining a clear picture of their own student assessment efforts, to provide a 
national profile of undergraduate student assessment efforts in postsecondary institutions, and to 
increase current understanding of how institutions can engage in and promote student assessment 
that produces positive impacts on academic, student, and institutional performance. A copy of the 
survey instrument is included in Appendix A. 

In January 1998, the ISSA instrument was mailed to the chief academic administrator at all 
postsecondary institutions recognized by the U.S. Office of Education that offer undergraduate 
programs at the associate or baccalaureate degree level, excluding proprietaiy and specialized 
institutions. Completed surveys were received from 1,393 of the 2,524 institutions meeting these 
criteria for an overall response rate of 55%. Associate of arts institutions were well represented in 
the survey responses. We received a total of 548 surveys from associate of arts institutions — 
more than half of the associate of arts institutions (54%) to whom instruments were mailed chose 
to complete and return the survey. In addition, more than half of eligible public and private 
associate of arts institutions participated, and survey responses were quite evenly distributed across 
states and accrediting regions. Given this strong and diversified response rate, we are able to offer 
a representative profile of undergraduate student assessment activities undertaken in associate of 
arts institutions. Throughout this report we compare student assessment activities within associate 
of arts institutions to student assessment activities within all responding institutions, including 
associate of arts institutions. 
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3. THE INFLUENCE OF EXTERNAL GROUPS 



Examines state and regional accreditation association requirements and 
external sources of support for student assessment. While both states and 
regional accrediting agencies have influenced institutions to engage in student 
assessment, accrediting associations are reported to have greater influence 
than states on associate of arts institutions ’ assessment initiatives. 



State officials and regional accreditation associations have increasingly required 
postsecondary institutions to become engaged in student assessment. Together with professional 
associations and private foundations, they have also offered various services to support 
institutions’ student assessment efforts. There has been relatively little examination of institutions’ 
perceptions of and experiences with these external groups. To examine this domain, our survey 
included questions concerning: state requirements for student assessment; regional accreditation 
association requirements for student assessment; and external sources of support for student 
assessment. The following sections address each of these areas in turn. 

State Role in Student Assessment 

States have varied widely in the development and content of their initiatives for 
postsecondary student assessment (Cole et al„ 1997). To examine the role of state influences in 
some detail, the survey asked for institutions’ perceptions of three dimensions of state assessment 
plans: the development process and reporting requirements of state assessment plans; the influence 
of state requirements on institutions’ assessment efforts; and state review of institutions’ 
assessment plans or processes. These questions were directed only to state-funded institutions. 

Development and Reporting Requirements of State Assessment Plans 



Development. Scholars have suggested that the influence of state assessment initiatives on 
institutions’ assessment efforts may depend on whether or not institutions have had input in the 
development of the initiatives. Survey respondents were asked to indicate whether their state’s 
plan for student assessment was developed primarily by state officials, through joint consultation 
between state officials and institutional representatives, or whether no state plan or requirement for 
student assessment existed. Responses from associate of arts and all institutions receiving state 
funding are displayed in Figure 2. 



Figure 2. Development of State Plan for Student Assessment 
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More than a third of the public associate of arts institutions report that there is no state plan 
for student assessment (38%). Of the remainder who say there is a state plan, most report that it 
was developed jointly by both state officials and campus representatives (45%). Almost one-fifth 
report that the state plan for student assessment was developed solely by state officials. Compared 
to all institutions, associate of arts institutions are slightly less likely to report that there is no state 
plan for assessment. 



Reporting _ Requirements . Scholars also contend that the influence of state assessment 
initiatives will vary with the specific nature of the reporting requirements associated with these 
initiatives. What types of information are institutions required to report to state officials? We 
asked institutions with state plans for assessment which of the following types of information they 
are required to report as part of their state’s assessment plan: evidence of a student assessment 
plan; measurement of state-mandated student performance indicators; measurement of 
institutionally-developed student performance indicators; and evidence of having used student 
assessment information. Institutions checked all applicable reporting requirements. Figure 3 
presents these responses for public associate of arts and all public institutions in our study. 

Figure 3. State Reporting Requirements for Student Assessment* 
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*Only institutions with a state plan for student assessment responded to this question. 



The most common reporting requirements for associate of arts institutions are evidence of a 
student assessment plan (69%) and measures of state-mandated student performance indicators 
(66%). Approximately half of respondents say they must report on institutionally-devised student 
performance indicators (48%) and are required to provide evidence of having used student 
assessment information (53%). The emphasis on evidence of an assessment plan and state- 
mandated student performance indicators is expected to contribute less to institutional support for 
student assessment than reporting requirements that emphasize evidence of assessment information 
use and institutionally-developed student performance indicators. There are no major differences 
in the state reporting requirements experienced by associate of arts institutions and all institutions. 

Influence of State Assessment Requirements 

What influence have state assessment requirements had on institutions’ undergraduate 
student assessment activities? Institutions were asked which of the following impacts on their 
student assessment activities they attribute to state assessment requirements: were an important 
reason for institution initiating undergraduate student assessment; increased institution’s 
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involvement in undergraduate student assessment; were not a factor in institution’s undergraduate 
student assessment activities, or were a negative influence on institution’s undergraduate student 
assessment activities. Institutions selected all applicable influences. Responses from public 
associate of arts and all public institutions are displayed in Figure 4. 

Figure 4. Influence of State Requirements for Student Assessment on Institutions’ 
Assessment Activities* 



100 



CT> 

c 

u 

o 

CL 

G) 

oc 

<n 

c 

o 



<n 

c 




reason to 
initiate 



increased 

involvement 



negative 

influence 



*Only institutions with a state plan for student assessment responded to this question 



Among associate of arts institutions reporting a state plan, most report that their state’s 
assessment plan has increased their involvement in student assessment activities (69%). Many 
report that their state’s mandate was an important reason for initiating their student assessment 
process (48%). Few respondents report that their state’s assessment mandate is not a factor in 
their student assessment efforts (15%) or that it has a negative influence on their efforts (4%). 
Compared to all public institutions, public associate of arts institutions are more likely to report that 
their state’s mandate has increased involvement in student assessment efforts on their campus. 

State Review of Institutions’ Student Assessment Plans 

Occurrence of State Review . States vary in terms of whether and how they evaluate 
institutions’ student assessment plans or information (National Center for Higher Education 
Management Systems, 1996). The survey asked respondents whether state officials had reviewed 
or evaluated their institution’s student assessment plans or processes. If a state review or 
evaluation had occurred, respondents indicated whether this review had been conducted by state 
officials, external reviewers, or the institution itself. Responses from public associate of arts and 
all public institutions are presented in Figure-5. 

Most public associate of arts institutions in states with mandated student assessment 
activities report that their student assessment plan or policy has been reviewed (64%). The most 
common form of review is by the state itself (46%). Approximately one-quarter of associate of 
arts institutions report that they have been required to do a self-review and the least common 
method for reviewing the institution’s student assessment policy or plan is by external reviewers 
(20%). Public associate of arts institutions are more likely to report that they have had a state 
review of their student assessment plan or policy than are all public institutions in our study. 

State Review Criteria . Institutions that have undergone a state review of their student 
assessment plan or process specified if the evaluation: reviewed the institution’s student 
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assessment process; compared the institution’s student performance record to its past performance; 
compared the institution s student performance record with that of peer institutions; or compared 
the institution’s student performance record with that of other institutions in the state. Responses 
from public associate of arts and all public institutions are presented in Figure 6. 

Figure 5. Methods Used to Review Institutions’ Student Assessment Plans or Processes* 
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*Only institutions with a state plan for student assessment responded to this question 



Figure 6. State Review Criteria for Institutions’ Student Assessment Plans or Processes* 
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For those associate of arts institutions that. had undergone a state-level review of their 
student assessment plan or process, approximately half report that the state reviewed the 
institution s student assessment process itself (48%). States also used comparisons in evaluating 
institutions’ student assessment processes. Comparisons with an institution’s own students’ past 
performance is most common (41%), followed by comparisons to peer institutions (38%) and 
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comparisons with other institutions in the same state (36%). Compared to all institutions, associate 
of arts institutions are more likely to be compared with peers. Since most states have more than 
one associate of arts institution, comparison with peers may be more feasible than it is for other 
types of institutions. 

Regional Accrediting Role in Student Assessment 

As noted earlier, regional accreditation agencies have been reported as important influences 
on institutions’ decisions to begin or expand their student assessment activities (El-Khawas, 1990, 
1992, 1995; Johnson et al., 1991). The majority of associate of arts institutions participating in 
our study (80%) have completed a regional accreditation review requiring undergraduate student 
assessment. To further examine the role of regional accreditation associations in student 
assessment, the survey asked about the reporting requirements for regional accreditation and 
institutions’ perceptions of the influence of regional accreditation requirements on their assessment 
efforts. 

Regional Accreditation Reporting Requirements for Student Assessment 

Regional accreditation associations vary in terms of the evidence they require institutions to 
report on student assessment efforts. It may be expected that requirements to report on the use 
being made of assessment information may contribute to greater institutional impacts from 
assessment than requirements that ask merely for evidence of an assessment plan or assessment 
data. We asked institutions which of the following types of information they were required to 
report to their regional accreditation association: evidence of a student assessment plan; intended 
institutional uses of student assessment information; results of student assessment; and evidence of 
having used student assessment information. Institutions checked all applicable reporting 
requirements. Very few (4%) of associate of arts institutions were unfamiliar with their regional 
accreditation requirements for student assessment. Responses from associate of arts and all 
institutions are displayed in Figure 7. 

Figure 7. Regional Accreditation Association Reporting Requirements for Student- 
Assessment 
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Most associate of arts respondents report that they are required to submit all four of these 
criteria to their regional accreditation agency. Most institutions are required to submit evidence that 
they have a student assessment plan in place (89%). The requirement to submit evidence of actual 
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institutional use of student assessment information (78%) is slightly more common than the 
requirement to submit only intended institutional uses (72%). While more than half of associate of 
arts institutions are required to submit the results of student assessment (66%), this requirement is 
least common among the four. There are no great differences between associate of arts and all 
institutions on these reporting requirements. 

Influence of Regional Accreditation Association Requirements for Student Assessment 

What influence have regional accreditation association requirements had on institutions’ 
undergraduate student assessment activities? Institutions were asked which of the following 
impacts on their student assessment activities they attributed to their regional accreditor’s 
assessment requirements: were an important reason for initiating undergraduate student 
assessment; increased institution’s involvement in undergraduate student assessment; were not a 
factor in institution’s undergraduate student assessment activities; or were a negative influence on 
institution’s undergraduate student assessment activities. Institutions selected all applicable 
influences. Responses from associate of arts and all institutions are displayed in Figure 8. 

Figure 8. Influence of Regional Accreditation Requirements for Student Assessment on 
Institutions’ Assessment Activities 
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Most responding associate of arts institutions report that regional accreditation requirements 
have increased their involvement with student assessment (75%). Many say that the requirements 
were an important reason to initiate a student assessment process (62%). Very few associate of 
arts institutional respondents report that regional accreditation requirements either have not 
influenced (15%) or have a negative influence on (0.8%) their student assessment process. 
However, associate of arts institutions seem to be slighdy less affected by regional accrediting 
associations than are all institutions. 

External Sources of Support for Student Assessment 

External constituents may influence institutions’ involvement in student assessment by 
providing funding or services to support assessment efforts. Little is known "about the extent to 
which institutions have availed themselves of these external sources of support. Our survey asked 
institutions about their use of external grants and other services for improving their student 
assessment efforts. 
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Receipt of External Grants for Student Assessment 

Institutions were asked if they had received grants to improve or support their student 
assessment practices from any of the following external sources: Fund for the Improvement of 
Postsecondary Education (FIPSE), other federal agencies, state incentive programs, and private 
foundations or corporate sources. Responses from associate of arts and all institutions are 
presented in Figure 9. 

Figure 9. Receipt of External Grants for Student Assessment by Source 
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Approximately one-fifth of responding associate of arts institutions have received an 
external grant for student assessment. Of the grants that were awarded to associate of arts 
institutions, most report receiving some type of federal grant other than FIPSE (10%), followed by 
state grants (8%) and private foundation or corporation and FIPSE grants (both 2%). All 
institutions are more likely to receive FIPSE and private foundation or corporate grants, while 
associate of arts institutions are more likely to receive other federal and state grants. 

Use of External Services to Support Student Assessment 

A variety of postsecondary organizations — professional associations, regional accrediting 
associations, state-level agencies, and consortia of institutions — provide a range of services 
intended to support institutions’ student assessment efforts. These services include consultation, 
assessment conferences, training workshops, and publications or research reports on student 
assessment. For each of these four types of postsecondary organizations, the survey asked 
institutions which, if any, of these student assessment services they had used. Responses from 
associate of arts and all institutions are presented in Table 1. 

While many associate of arts institutions report that external services are not used or not 
available, they do use assessment-related services from some types of postsecondary organizations 
more than others. They are more likely to use services from professional and regional accreditation 
associations than from state agencies or institutional consortia. They are also likely to use certain 
types of assessment support services more than others, regardless of service provider. Associate 
of arts institutions make most frequent use of publications or research reports and conferences on 
student assessment. A smaller proportion has used training workshops from one or more 
providers while the smallest proportion has used consultation services. Compared to all 
institutions in our study, associate of arts institutions make greater use of state-provided training 
workshops on assessment. 
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Table 1. Institutional Use of External Services to Support Student Assessment by Service 
Provider 

% of Institutions Using 
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Summary Observations 

Both states and regional accrediting agencies have influenced institutions to engage in 
student assessment. Accrediting associations were reported to have greater influence than states in 
stimulating associate of arts institutions to initiate involvement in student assessment. Despite the 
obtrusiveness of these two external bodies, institutions tend to view their mandates and 
requirements as having a positive influence on their student assessment activities. Part of this 
positive image may stem from the services provided by states and accreditation associations which 
these institutions are using to support their student assessment efforts. While institutions are 
taking advantage of external resources, they are, for the most part, not receiving external funding 
to support their endeavors. Perhaps funding agents believe that assessing students should be a 
function of the institutional budget, rather than a special grant-funded project. 

Somewhat surprising is the number of associate of arts institutions that have undergone a 
review of their student assessment plan or process. A majority of institutions have either gone 
through an accreditation review or a state review (or both) of their assessment process. Both of 
these review processes go beyond merely reviewing the existence of an assessment plan. Many 
states have systems in place to compare an institution’s student assessment outcomes to either past 
student outcomes or to those of a peer institution. Many accreditation bodies seek evidence that 
student assessment information is used in decision-making. The student assessment movement 
has advanced to the point where almost every institution should have had its student assessment 
process reviewed by at least one external body. 

Compared to all institutions, associate of arts institutions are more likely to be involved 
with their state on student assessment initiatives. They are more likely to say that their state 
mandate increased their involvement with assessment, they are more likely to receive a grant from 
their state, and they are more likely to take advantage of workshops on student assessment 
provided by the state. Associate of arts institutions are also more likely to be public, which may 
account for these differences. 
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4. APPROACHES TO STUDENT ASSESSMENT 



Addresses the type or content of student assessment measures used by 
institutions, the timing of those assessments, their use of standardized 
instruments and less traditional student assessment methods, and types of 
assessment studies conducted and student performance reports produced. 
Associate of arts institutions are not fully engaged with student assessment at 
this point in time, but most often collect data on students’ basic college- 
readiness skills, their academic intentions, and their academic progress, and 
least often collect data on higher-order skills, students’ personal growth, and 
former students ’ civic or social roles. 

Assessment approach decisions may be influenced by external mandates, institutional 
context, and the organizational and administrative environment. An institution’s assessment 
approach may shape the uses and impacts of assessment information. Assessment scholars 
contend that institutions which collect comprehensive student assessment information — those 
which collect data on various dimensions of student performance, at multiple points in time, 
through a variety of assessment methods — and which conduct and report a variety of analyses of 
assessment data are more likely to use and achieve positive impacts from student assessment data. 

What approaches to student assessment have associate of arts institutions adopted? The 
ISSA instrument addressed the type or content of student assessment measures used by 
institutions, the timing of those assessments, their use of standardized instruments and less 
traditional student assessment methods, and types of assessment studies conducted and student 
performance reports produced. Here we examine the profile of student assessment approaches in 
associate of arts institutions and compare it to the profile for all types of postsecondary institutions. 

Type and Extent of Student Data Collected 

Data Collected on Current Students 



What types of data do institutions collect from their students? Survey respondents reported 
the extent to which their institution collects data on ten types of student data for currently enrolled 
students and four measures for former students. Figures 10, 1 1, and 12 display mean scores for 
the extent to which associate of arts institutions and all institutions collect these data. 

Cognitive Data. Figure 10 shows the extent to which institutions collect five types of 
cognitive student assessment data: basic college-readiness skills, higher-order skills (e.g., critical 
thinking and problem solving), general education competencies, competence in major field of study 
(discipline- or program-specific knowledge), and vocational or professional skills. The latter four 
of these five types of data factored together to create a “cognitive assessment” index that we used in 
regression analyses presented in section eight. 

Associate of arts institutions collect the most data on students’ basic college-readiness skills 
(3.56), collecting these data for “many” students. This finding is not surprising, given the open 
admissions policies of most associate of arts institutions. Conversely, associate of arts institutions 
tend to collect less data on higher-order skills (1.88), general education competence (2.41), 
competence in the students’ major (2.30), and vocational skills (2.25), collecting these data only 
on “some” students. Compared to all institutions, associate of arts institutions are more likely to 
collect data on both basic and vocational skills. They are less likely to collect cognitive data on 
higher-order skills, general education skills, and competence in the students’ major. 



O 

ERIC 



19 



27 



Figure 10. Extent of Cognitive Data Collected on Current Students 




| COGNITIVE INDEX 1 

l=not collected; 2=collected for some students; 3=collected for many students; 4=collected for all students. 



Affective and Behavioral Data . Figure 1 1 presents five types of affective and behavioral 
data collected on currendy enrolled students: personal growth and affective development, student 
experiences and involvement with the institution, student satisfaction with the institution, students’ 
academic intentions, and students’ academic progress. The first three of these items factored 
together into an “affective assessment” index that we used in regression analyses presented in 
section eight. 

Figure 11. Extent of Affective and Behavioral Data Collected on Current Students 




affective involvement satisfaction intentions progress 

I AFFECTIVE INDEX 1 

l=not collected; 2=collected for some students; 3=collected for many students; 4=collected for all students. 



Associate of arts institutions are most likely to collect data on students’ academic progress 
(3.58). This finding parallels the proclivity of these institutions to measure students’ academic 
intentions (3.38). Because of the varied student goals at associate of arts institutions, one way to 
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measure student success is to discern students’ goals and then to track students, ensuring that they 
are meeting their goals. Nonetheless, more institutions measure students’ academic progress than 
their academic intentions. Although one would think that it is necessary to first discern students’ 
academic intentions before evaluating their academic progress, it is probably easier to monitor 
academic progress via transcripts and other institutional data than it is to collect and update 
information on students’ goals. 

Associate of arts institutions are less likely to measure either students’ satisfaction (2.86) or 
their involvement (2.35) with the institution. They are least likely to collect data on students’ 
personal and affective growth ( 1 .77). These institutions are less likely to collect data on students’ 
personal growth than on any other type of student outcome listed on our instrument for current 
students. Perhaps associate of arts students are not at the institution long enough to enable 
measuring growth over time. Furthermore, most associate of arts students do not live on campus 
and it would be difficult to attribute the personal growth they experience over any given time period 
to their interaction with the institution. Compared to all institutions in our study, associate of arts 
institutions are less likely to collect data on students in all these affective/behavioral areas, except 
for academic intentions. 



Data Collected on Former Students 



Figure 12 displays mean scores for the extent to which associate of arts and all institutions 
collect four types of data from former students: civic or social roles (e.g., political, social, or 
community involvement), satisfaction and experiences with the institution after leaving, further 
education (e.g., transfer, degree attainment, graduate study), and vocational or professional 
outcomes (e.g., job attainment or performance). These latter three items factored together to create 
a “post-college” index used in regression analyses presented in section eight. 



Figure 12. The Extent of Data Collected on Former Students 
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POST-COLLEGE INDEX 1 



l=not collected; 2=collected for some students; 3=collected for many students; 4=collected for all students. 



Associate of arts institutions collect data on “some” to “many” students on vocational 
outcomes (2.73), further education (2.71), and their satisfaction (2.57). They are least likely to 
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collect data on students’ post-college civic or social roles (1.37). Most associate of arts institutions 
do not collect this data, perhaps believing that students spend too little time on their campuses to 
warrant linking post-college civic roles to their campus experiences. Compared to all institutions in 
our study, associate of arts institutions are much less likely to measure post-college civic roles than 
are all institutions in our study. 

Timing of Data Collection 

Researchers have stressed that it is particularly important for institutions to collect student 
assessment data on students at varying points in time, in order to assure that students are meeting 
their academic goals and that the college experience is “value-added.” When are associate of arts 
institutions collecting student assessment data? Table 2 presents the timing of the student 
assessment data collection efforts for each of nine measures of student performance for associate of 
arts and all institutions. Institutional respondents told us whether they collect such data at entry, 
during student enrollment, and/or at exit. Respondents could choose all applicable responses. 

Timing of Collecting Cognitive Data 

With the exception of basic skills data, most cognitive data are collected either while the 
student is enrolled or at exit. It is not surprising that most associate of arts institutions which 
collect basic skills data do so at the point of entry into the institution (98%). However, only one- 
fifth of these institutions report that basic skills are assessed while the student is enrolled. More 
than half of the associate of arts institutions which collect data on a student’s competence in the 
major do so while the student is enrolled (61%) and again at exit (59%). While this finding seems 
to indicate that students are assessed at more than one point in time, in actuality, less than half 
(42%) of associate of arts institutions are collecting major competence data on students both while 
enrolled and at exit — the remaining institutions are collecting major competence data either while 
the student is enrolled or at exit. There are no great differences between associate of arts and all 
institutions on their timing of data collection. 



Table 2. Institutions Collecting Cognitive, Affective, and Behavioral Student Assessment 
Data at Entry, While Student is Enrolled, and at Exit 

% Institutions Collecting Data 

At Entry While Enrolled At Exit 

Data Collected AA All AA All AA All 



Cognitive Data 



Basic Skills 


98 


97 


19 


21 


6 


8 


Higher Order Skills 


24 


32 


63 


66 


33 


33 


General Education 


30 


32 


56 


64 


40 


36 


Major Competence 


12 


9 


61 


62 


59 


64 


Vocational Skills 


14 


10 


63 


67 


61 


60 


Affective and Behavioral 














Data 














Personal Growth 


16 


32 


72 


67 


33 


41 


Student Involvement 


4 


7 


71 


72 


51 


55 


Student Satisfaction 


- - 4 


7- ■ 


69 


-7-1 


... —62 


65 


Academic Intentions 


94 


92 


32 


36 


14 


19 
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Timing of Collecting Affective and Behavioral Data 

With the exception of academic intentions, most affective data are collected either during 
enrollment or at exit. In terms of academic intentions, it again is not surprising that students’ 
intentions are collected at entry (94% collect these data at entry). It seems apparent that these 
intentions are not assessed again, even though students’ goals may change throughout their time at 
an institution. Compared to all institutions, associate of arts institutions are less likely to develop a 
benchmark of students’ personal growth at entry. 

Student Assessment Instruments and Methods Used 

Institutions engaging in student assessment efforts must select the means by which 
assessment data will be collected. A traditional choice is to use assessment instruments, generally 
in the form of objective examinations or inventories administered in a pencil and paper or 
computerized format. More recently, there has been growing interest in alternative methods of 
assessing students. These non-traditional methods tend to be more qualitative or integrative in 
approach — requiring students to demonstrate higher-order cognitive skills such as application or 
synthesis — or may use sources of information other than students themselves. Further, as the 
student body entering postsecondary institutions becomes more diverse through the increased 
enrollment of part-time, older, and minority students), some scholars have suggested the need to 
use special assessment methods to reflect the unique characteristics, learning styles, needs, and life 
situations of various sub-populations of students. 

Student Assessment Instruments 

How are institutions collecting student assessment data? Survey respondents reported 
whether their institution used instruments or tests'to collect any of ten types of assessment 
information: nine for current students (basic college-readiness skills; higher-order skills; general 
education competencies; competence in major field of study; vocational or professional skills; 
academic intentions; personal growth and affective development; experiences and involvement with 
the institution; and satisfaction with the institution) and one for former students (alumni satisfaction 
and experiences). If an instrument is used, respondents specified its source — institutionally- 
developed, state-provided, or commercially available. Figures 13 and 14 present information on 
these ten types of assessment information collected via traditional instruments or tests by associate 
of arts and all institutions. 

Figure 13. Institutional Use of Student Assessment Instruments to Collect Cognitive Data 
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Figure 14. Institutional Use of Student Assessment Instruments to Collect Affective and 
Behavioral Data 




intentions affective involvement satisfaction satisfaction 



As Figures 13 and 14 show, the use of assessment instruments or tests varies with the type 
of assessment data collected. Associate of arts institutions are most likely to use instruments or 
inventories to collect data on basic college-readiness skills (99%), student (96%) and alumni (84%) 
satisfaction, and student intentions (82%). These institutions are least likely to use instruments or 
inventories to collect data on students’ personal growth and affective development (46%), higher- 
order cognitive skills (49%), general education competencies (65%), and competence in the major 
(70%). Compared to all institutions, associate of arts institutions are less likely, in general, to use 
instruments to assess students. Specifically, they are less likely to use instruments to measure 
personal growth, higher-order skills, students’ experiences, competencies in the major field, 
general education competencies, and alumni satisfaction. Associate of arts institutions are more 
likely to use instruments to measure basic skills, vocational skills, and students’ academic 
intentions. 

Other Student Assessment Methods 

Institutions also reported the extent to which they use each of nine alternative methods of 
assessing students. Table 3 presents this information for associate of arts and all institutions. 

Four of these options require more active participation of currently enrolled students and factored 
together to create a “student-centered” index of assessment methods that we used in regression 



Table 3. Mean Scores of Extent of Use of Other Student Assessment Methods 



Student Assessment Methods 


Associate of Arts (n=539) 


All Institutions (n=1393) 


Student-Centered 






Observations of student performance 


2.22 


2.26 


Student portfolios or comprehensive projects 


1.95 


2.10 


Student performance in capstone courses 


1.78 


2.15 


Student interviews or focus groups 


1.65 


1.84 


Extemallv Oriented 






Employer interviews or focus groups 


1.98 


1.87 


Alumni interviews or focus groups ~ ~ 


1.80 ~ 


1.90 


Other Methods 






Surveys or interviews with withdrawing students 


2.26 


2.40 


Transcript analysis 


2.19 


2.16 


External examination of students 


2.01 


2.02 



1 = not used; 2 = used in some units; 3 = used in most units; 4 = used in all units 
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analyses presented in section eight: observations of student performance, student portfolios or 
comprehensive projects, student performance in capstone courses, and student interviews or focus 
groups. None of these four methods is used extensively by associate of arts institutions. They are 
most likely to use observations of student performance (2.22). The other three student-centered 
methods are either not used by associate of arts institutions or only used in some units. The least 
often used of these three methods is interviews or focus groups with students (1.65). 

Two of the other alternative measures rely on information from external groups or sources 
and factored together to create an index of “external methods” of student assessment that we used 
in regression analyses in section eight: alumni interviews or focus groups, and employer 
interviews or focus groups. Associate of arts institutions are more likely to interview or hold focus 
groups with employers (1.98) than with alumni (1.80), which parallels associate of arts 
institutions’ traditionally strong ties to their employer communities. 

There were three additional alternative student assessment measures listed in our survey: 
transcript analysis, using external examinations, and surveying or interviewing withdrawing 
students. Of these three, associate of arts institutions are most likely to survey/interview 
withdrawing students (2.26). They are least likely to use external examiners or examinations of 
students and their use of transcript analysis falls in the middle of these two. 

Compared to all institutions in our study, associate of arts institutions are less likely to use 
all four of the student-centered methods for collecting assessment data. Regarding external 
methods, for all institutions in our study, alumni interviews/focus groups are more common than 
employer interviews/focus groups, which is the opposite pattern for associate of arts institutions. 
For the remaining alternative methods, all institutions are more likely to survey or interview 
withdrawing students than are associate of arts institutions. 

Assessment Methods for Student Sub-Populations 

Do institutions use different assessment methods for specific sub-populations of their 
student body? Table 4 presents the percentage of associate of arts and all institutions using 
different assessment methods for four student sub-populations: adult students, part-time students, 
minority students, and distance education students. 



Table 4. Institutional Use of Different Assessment Methods for Different Student Sub- 
Populations 



Student Sub-Population 


% Institutions Using Different Assessment Methods 


Associate of Arts (n=545) 


All Institutions (n=1366) 


Adult students 


3 


10 


Part-time students 


3 


5 


Minority students 


2 


2 


Distance education students 


21 


22 



Most associate of arts institutions do not use different assessment methods for specific sub- 
populations of undergraduate students. While 21% report using different methods for distance 
education students, a decision that may be necessitated by the medium used for course delivery, 
very few use special methods for collecting assessment data from their adult, part-time, or minority 
students. This finding is likely reflective of the populations attending associate of arts 
institutions — often part-time or adult students comprise the majority of, rather than a specific sub- 
population of students. Therefore, it is not surprising that when compared to all institutions, 
associate of arts institutions are even less likely to use different assessment measures for adult 
students (3% vs. 10%). 
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Student Assessment Studies and Reports 

Beyond collecting descriptive data on their students’ characteristics, performance, or 
development, institutions must consider how to transform the data into useful information. 
Institutions that study the relationship between various aspects of students’ institutional 
experiences and their performance should be better able to make informed decisions concerning 
academic and student-related practices and policies. Another consideration is the level of 
aggregation at which assessment information is analyzed and reported. 

Student Assessment Studies 

Are institutions studying how student performance is connected to experiences with the 
institution? In our survey, we asked if institutions conduct studies of the relationship between nine 
areas of students’ institutional experiences and students’ performance. Table 5 identifies these 
areas and presents data on institutional studies for both associate of arts and all institutions. 

Many associate of arts institutional respondents (45%) report that they do not conduct any 
of these studies on students’ experiences. For those that do conduct studies, they are most likely 
to study relationships with student financial aid and/or concurrent employment (27%); admissions 
standards or policies (27%); student course-taking patterns (26%); exposure to different 
instructional or teaching methods (26%); and academic advising patterns (24%). They are least 
likely to conduct studies on residence arrangements (6%), patterns of student-faculty interaction 
(11 %), and extra-curricular experiences ( 1 5%). 

Table 5. Institutions Conducting Student Assessment Studies 

% Institutions Conducting Studies 

Studies of Relationship Between Student Performance " Associate of Arts All Institutions 

and the Following Experiences (n=519) (n= 1 329) 



Do not study these experiences 


45 


38 


Student financial aid and/or concurrent employment 


27 


30 


Admission standards or policies 


27 


42 


Student course-taking patterns 


26 


26 


Exposure to different instructional or teaching methods 


25 


21 


Academic advising patterns 


24 


26 


Classroom, library and/or computing resources 


19 


17 


Extra-curricular activities 


15 


24 


Patterns of student-faculty interaction 


11 


14 


Residence arrangements 


6 


21 



Compared to all institutions, associate of arts institutions are slightly less likely to conduct 
studies on student-faculty interaction. While associate of arts institutions are less likely to conduct 
studies in general, most other comparisons demonstrate that the pattern of conducting studies is quite 
similar for associate of arts institutions and for all institutions, except for where differences would be 
expected, such as in studying either resident arrangements or admissions standards or policies. 

Student Assessment Reports 

Finally, institutions were asked to report the levels of aggregation at which they provide 
reports of student assessment information. Figure 15 shows the percentage of associate of arts and 
all institutions providing student assessment reports at five levels Of aggregation: institution- wide; 
school or college; program; special population of students; and course or groups of courses. 
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Respondents indicated as many levels of aggregation as were applicable or could indicate that they 
provide no reports of student assessment information. 

Figure 15. Preparation of Student Performance Reports by Level of Aggregation 



ioost 




institution school program subgroup course no reports 



Most associate of arts respondents prepare reports on student assessment results (87%). 
The most prevalent level of aggregation used is institution-wide (66%), followed closely by 
academic program/department (6 1 %). The least prevalent level of aggregation for associate of arts 
institutions is by school or college (13%). Compared to all institutions, associate of arts 
institutions are less likely to provide student assessment data at the school or college level, likely 
because these institutions are not typically organized into discrete schools or colleges. . 

Summary Observations 

Associate of arts institutions are more likely to collect data on students’ basic college- 
readiness skills, their academic intentions, and their academic progress than on any other types of 
student skills or outcomes. Conversely, associate of arts institutions are least likely to collect data 
on higher-order skills, students’ personal growth, and former students’ civic or social roles. All 
types of student assessment data are most likely to be collected at only one point in time. In 
collecting student data, associate of arts institutions are more likely to use traditional tests and 
instruments than to engage in more student-centered methods. Slightly more than half of all 
responding associate of arts institutions said they conduct studies on students linking their 
performance to their interaction with the institution, and a great majority of respondents provide 
reports on the results of student assessment initiatives. However, it appears that, on average, 
associate of arts institutions are not fully engaged with student assessment at this point in time, 
given the limited types of assessment data collected, the limited points in time of the data collection, 
and the limited number of studies connecting student performance to institutional experiences. 

Further evidence for this Finding is found in comparing associate of arts to all institutions. 
Associate of arts institutions are less likely to collect cognitive and affective data, less likely to use 
student-centered methods in collecting data, and less likely to conduct studies linking student 
performance to students’ interactions with their institution. Perhaps because associate of arts 
students are more likely to be attending part-time and to be commuting, it is more difficult to 
engage them in assessment activities. 
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5. ORGANIZATIONAL AND ADMINISTRATIVE SUPPORT PATTERNS 



Focuses on the patterns of organizational and administrative support that 
institutions have developed to promote student assessment on their 
campuses, such as their mission and purpose, institution-wide activities, 
leadership support, and planning and coordinating mechanisms. Associate 
of arts institutions’ patterns are varied and reflect that many of these support 
processes are administratively, rather than faculty, driven. 

An important focus of our study was to examine the patterns of organizational and 
administrative support institutions have developed to promote student assessment on their 
campuses. The assessment literature suggests that institutions will be most likely to promote 
internal support for assessment if they engage in assessment for internal rather than external 
purposes, have visible and strong leadership support for assessment, include a broad range of 
internal participants — particularly faculty — in making assessment-related decisions, and 
regularly evaluate their assessment programs. It is expected that organizational and administrative 
support will shape the assessment approach, assessment management practices and policies, and 
ultimately^ the institutional uses and impacts of assessment information. 

The ISSA instrument included questions concerning the institution-wide assessment 
support strategy; administrative and governance activities used to promote student assessment; 
leadership and faculty support for assessment; planning and coordination for assessment; and 
evaluation of student assessment processes. Once again, we will present information concerning 
the pattern of organizational and administrative support for student assessment in associate of arts 
institutions and compare it to the pattern for all types of postsecondary institutions. 

Institution-Wide Assessment Support Strategy 

Mission Emphasis 

Assessment scholars and practitioners suggest that institutions’ academic mission content 
may symbolize the importance with which student assessment is regarded and so may influence the 
degree of internal support for assessment. What do the mission statements of our responding 
institutions tell us about their values regarding student assessment? Institutions reported whether 
their mission statement explicitly: a) emphasizes excellence in undergraduate education, b) 
identifies intended student outcomes, c) refers to student assessment as an important priority, or d) 
does not mention any of these. Respondents could answer “yes” to more than one mission 
statement component. Figure 16 displays the percentage of associate of arts and all institutions 
reporting each of these mission statement components. 

Most associate of arts institutions emphasize excellence in undergraduate education in their 
mission statements (75%) and half identify intended student outcomes. Approximately one-fifth of 
associate of arts institutions refer to student assessment as an important activity in their mission 
statement. Compared to all institutions, associate of arts institutions are less likely to emphasize 
excellence in undergraduate education in their mission statements 

Purposes for Engaging in Student Assessment . . 

Why are institutions engaging in student assessment? A second dimension of institution- 
wide assessment support examined in this study is the purpose(s) underlying institutions’ student 
assessment efforts. Scholars contend that whether assessment is primarily engaged in for internal 
or for external purposes may influence the nature of an institution’s assessment approach, degree 
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Figure 16. Institutional Mission Emphasis 
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of internal support, and assessment uses and impacts. Institutions rated the importance of six 
purposes for engaging in student assessment: preparing for an accreditation self-study; meeting 
state reporting requirements; guiding internal resource allocation decisions; guiding undergraduate 
academic program improvement; improving the achievement of undergraduate students; and 
improving faculty instructional performance. The last four of these purposes were factored into an 
index of internal purposes for student assessment for use in regression analyses presented in 
section eight. Figure 17 presents the mean scores for each student assessment purpose for 
associate of arts and all institutions. 



Figure 17. Purposes of Student Assessment 




accreditation state reports allocate 
self-study resources 



academic undergrad instructional 
programs improvement improvement 



I Internal Purposes 1 

Scale: l=no importance; 2=minor importance; 3-moderate importance; 4=very important 



Associate of arts institutions report that preparing a self-study for accreditation is the most 
important reason to engage in student assessment (3.61), followed by improving the achievement 
of undergraduate students (3.50), guiding undergraduate academic program improvement (3.38), 
and meeting state reporting requirements (3.37). All are rated “moderate” to “very” important. 
They are less likely to say that improving faculty instructional performance (3.06) or guiding 
internal resource allocation decisions (2.83) is an important reason for engaging in student 
assessment. 
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Compared to all institutions, associate of arts institutions are more likely to report that 
meeting state requirements is an important reason to engage in student assessment. A greater 
percentage of associate of arts institutions are public — likely explaining this difference. A smaller 
percentage of associate of arts institutions are likely to say that preparing for a self-study is an 
important reason to engage in student assessment. 

Institution-Wide Administrative and Governance Activities 



Institutions may develop a variety of administrative activities and governance structures to 
promote student assessment on an institution-wide basis. How prevalent are these practices? The 
ISSA instrument asked whether institutions use any of the following administrative activities and 
governance structures: annual institution-wide initiatives on assessment; rewards or incentives for 
academic and student affairs administrators to use student assessment in their units; incentives for 
academic units to use assessment information in evaluation and improvement efforts; student 
assessment workshops for academic and student affairs administrators; board of trustees committee 
that addresses student assessment; faculty governance committee that addresses student assessment 
issues; and student representation on student assessment committees. Figure 18 displays the 
percentage of associate of arts and all institutions that have introduced these activities or structures 
in their institutions. 



Figure 18. Institution-Wide Administrative and Governance Activities Used to Promote 
Student Assessment 
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None of these seven activities is used pervasively by associate of arts institutions. 
Institutions are most likely to provide assessment workshops for academic and student affairs 
administrators (57%), to have a faculty governance committee that addresses assessment issues 
(50%), and to present annual institution-wide student assessment forums (48%). They are less 
likely to have student representation on assessment committees (28%) or to provide incentives for 
academic units to use assessment information in evaluation and improvement efforts (24%). They 
are least likely to have a board of trustees committee that addresses assessment (10%) or to provide 
rewards or incentives for academic and student affairs administrators who promote units’ use of 
assessment (4%). — - — — - - — - — — - 

Compared to all institutions, associate of arts institutions are less likely to have a faculty 
governance committee that addresses assessment issues. Associate of arts institutions are also less 
likely to have student representation on assessment committees, perhaps due to the greater 
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percentage of part-time students found at these institutions. Associate of arts institutions, on the 
other hand, are more likely to hold annual institution-wide student assessment forums. 

Leadership and Faculty Support for Student Assessment 

The degree to which student assessment efforts are supported internally is likely an 
important determinant of the extent and impact of an institution’s student assessment activities. 
How supportive of student assessment are various internal constituencies? We asked survey 
respondents to rate the degree to which six positions, or groups support undergraduate student 
assessment activities: board of trustees; chief executive officer; academic affairs administrators; 
student affairs administrators; faculty governance; and students. Figure 19 presents the mean 
responses to this question for associate of arts and all institutions. 

Institutions report high levels of support for student assessment, with most constituents at 
least somewhat supportive. Associate of arts respondents report that academic affairs 
administrators are most supportive (4.64), followed by the chief executive officer (4.47) and 
student affairs administrators (4.38). Boards of trustees (3.87) and faculty governance bodies 
(3.87) are slightly less supportive, while student support is seen as neutral or unknown (3.40). In 
all cases, associate of arts institutions’ constituents are reported as either equally or more 
supportive than constituents from all institutions in the study. 

Figure 19. Internal Constituent Support for Student Assessment 




affairs affairs governance 



l=very unsupportive; 2=somewhat unsupportive; 3=neutral, unknown; 4=somewhat supportive; 5=very supportive 



Planning and Coordinating Student Assessment 

A central issue in discussions of leadership and governance for student assessment 
concerns the degree to which responsibility for planning and coordinating student assessment 
activities is centralized or decentralized within institutions. Centralized approaches that focus such 
responsibility on senior administrators or central offices may signal that assessment is a valued 
institutional activity, but decentralized approaches involving an array of internal participants may 
do more to promote broader internal support for assessment. How do institutions plan and 
coordinate their assessment efforts? We asked institutional respondents about six aspects of their 
assessment planning and coordination process: the nature of an institutional plan or policy for 
student assessment; membership on an institution-wide planning group for assessment; executive 
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responsibility for the assessment planning process; approval authority for assessment plans; 
operational responsibility for student assessment; and reporting patterns. 

Institutional Plan or Policy for Student Assessment 

We asked survey respondents which of seven types of institutional plans or policies for 
student assessment best describe what exists at their institution: 1) formal comprehensive 
centralization — a formally adopted plan or policy specifying undergraduate student assessment 
activities for all academic programs or units; 2) formal limited centralization — a formally adopted 
plan or policy for undergraduate student assessment in some academic programs or units; 3) formal 
decentralization — a formally adopted institutional plan or policy requiring all academic units or 
programs to develop their own undergraduate student assessment plan; 4) formal guidance — a 
formally adopted institutional plan or policy identifying institution-wide activities to be conducted 
by a central committee or office; 5) informal encouragement — no institutional plan or policy but 
academic units or programs are encouraged to develop their own undergraduate student assessment 
activities; 6) emergent — institution is currently developing a plan or policy for undergraduate 
student assessment; and 7) none — institution does not have an undergraduate student assessment 
plan or policy. Respondents could select more than one type of plan or policy. Figure 20 presents 
the percentage of associate of arts and all institutions using each type of assessment plan or policy. 

Figure 20. Types of Institutional Plan or Policy for Student Assessment 




Virtually all associate of arts institutions report that they have a plan or policy for student 
assessment; only 4% have no policy or plan. Associate of arts institutions are most likely to report 
that they have a formal centralized student assessment policy (54%), followed by a formal 
guidance policy (38%). They are less likely to have any of the other types of plan or policy. The 
greatest difference between associate of arts and all institutions is that all institutions are more likely 
to have a formal decentralized policy. Since associate of arts institutions often have more 
centralized administrative governance patterns and are less likely to have discrete operating units, 
this finding is not surprising. In fact, associate of arts institutions are more likely than all 
institutions in our study to have the most formal and most centralized plan or policy for student 
assessment. 
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Institution-Wide Assessment Planning Group Membership 

Who is involved in student assessment planning? We asked respondents whether they 
have an institution-wide group responsible for ongoing planning and policy setting for 
undergraduate assessment. Fully 73% of associate of arts institutions and 70% of all institutions 
report having such a group. Of those institutions with institution-wide planning groups, we asked 
which of the following internal constituents serves on the group: chief executive officer, academic 
affairs administrator or staff, student affairs administrator or staff, institutional research 
administrator or staff, academic review and evaluation administrator or staff, student assessment 
administrator or staff, faculty, and students. Respondents checked as many constituents as were 
applicable. Figure 2 1 presents the percentage of associate of arts and all institutions responding to 
whether each constituent is a member of their assessment planning group. 

For associate of arts institutions with a planning group, student assessment planning 
committees are staffed by faculty (in 91% of the institutions) and academic administrators (86%). 
They also are likely to have student affairs administrators (67%) and institutional research 
personnel (67%) on their planning committees. They are less likely to have student assessment 
staff (37%), students (28%), academic review or evaluation staff (24%), or the chief executive 
officer (17%) involved. Compared to all institutions, associate of arts institutions are more likely 
to have student affairs staff, institutional research personnel, and student assessment staff on their 
planning committees, but are less likely to include students. 



Figure 21. Membership on Institution-Wide Student Assessment Planning Group* 
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♦Only institutions with institution-wide planning groups for student assessment responded to this question 



Executive Responsibility for Assessment Planning Process 

Another indicator of the governance process used for student assessment is where formal 
leadership for assessment planning is vested in an institution. We asked institutions which of the 
following positions has executive responsibility for or chairs the institution-wide group for student - 
assessment planning: academic affairs administrator; student affairs administrator; institutional 
research officer; academic review and evaluation officer; student assessment officer; or faculty 
member. Figure 22 presents this information for associate of arts and all institutions. 
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Figure 22. Executive Responsibility for Student Assessment Planning Group 
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At more than half of associate of arts institutions, academic administrators have 
responsibility for the student assessment planning group (54%), followed by a faculty member 
(30%). Less than 20% of the institutions have vested an institutional research officer or a student 
affairs administrator with executive responsibility for the student assessment planning group. It is 
very rare for either academic review (3%) or student assessment administrators (8%) to have 
executive responsibility for the student assessment planning group at associate of arts institutions. 
The pattern is similar for all institutions. There is a modest difference between associate of arts and 
all institutions in that associate of arts institutions are more likely to appoint a student affairs 
administrator to lead student assessment initiatives. 

Approval Authority for Student Assessment Plan 

As a final aspect of governance for assessment planning, we asked institutions to identify 
who, among ten possible positions or groups, approves changes in their institutional plan or policy 
for student assessment: board of trustees; chief executive officer; chief academic affairs officer; 
chief student affairs officer; institutional research officer; academic review and evaluation officer; 
student assessment officer; student government; academic senate or other faculty committee; and 
faculty union. All respondents could answer this question regardless of whether or not they had an 
institution-wide committee for assessment planning. Respondents indicated as many positions or 
groups as were applicable. Only 1 1 % of associate of arts institutions report that academic review 
and evaluation officers, student government, or faculty union personnel have approval authority 
for the institutional assessment plan or policy. Figure 23 presents the results for the remaining 
seven positions for associate of arts and all institutions. 

Over 70% of associate of arts institutions report that the chief academic officer has approval 
authority for student assessment. The chief executive officer has such authority in approximately 
half the institutions. Associate of arts institutions are not likely to invest approval authority in a 
student assessment officer (12%), an institutional researcher (20%), the board of trustees (24%), a 
chief student affairs officer (28%), or an academic senate (30%). Compared to all institutions, 
associate of arts institutions are more likely to invest the chief executive officer, the board of 
trustees, or the chief student affairs officer with approval authority. Boards at associate of arts 
institutions may be more involved with teaching and learning issues than are boards at other 
institutions. Conversely, associate of arts institutions are less likely to invest the chief academic 
officer or the academic senate with student assessment approval authority. 
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Figure 23. Approval Authority for Student Assessment Plan or Policy 
100 -/ 



o> 

C 

r 

o 

CL 

<D 

DC 

(/) 

c 

o 



(/> 

c 




trustees CED chief chief student 

academic affairs 
affairs 



student 

assessment 



academic 

senate 



Operational Responsibility for Day-to-Day Student Assessment Activities 



Beyond the planning phase of student assessment, institutions must decide where to place 
responsibility for overseeing the day-to-day undergraduate student assessment activities such as 
instrument development, data collection, analysis, and reporting. Where have institutions placed 
operational responsibility for student assessment? We asked institutions which of six positions or 
offices has responsibility for day-to-day student assessment activities: academic affairs 
administrator; student affairs administrator; institutional research officer; academic review and 
evaluation officer; student assessment officer; and faculty member(s). Institutions checked as 
many positions/offices as applied. Figure 24 presents the results for associate of arts and all 
institutions. 



Figure 24. Operating Responsibility for Day-to-Day Student Assessment Activities 








At associate of arts institutions, either an institutional research officer (49%) or an academic 
affairs administrator (42%) is most likely to be responsible for day-to-day operational 
responsibility for student assessment. Less frequently, an assessment officer (18%), a student 
affairs administrator (24%), or a faculty member (27%) is identified. These institutions are least 
likely to have an academic review officer in charge of day-to-day activities (6%). Compared to all 
institutions, associate of arts institutions are more likely to have an institutional research officer, a 
student affairs administrator, or an assessment officer in charge of day-to-day responsibilities. 
Conversely, associate of arts institutions are less likely to have an academic affairs administrator, 
an academic review officer, or a faculty member in charge of day-to-day activities. 

Reporting Relationship of Individual with Operating Responsibility for Student Assessment 

Finally, institutions were asked to whom the individual with day-to-day operating 
responsibility for student assessment reports. Survey respondents selected from the following five 
offices: chief executive officer, chief academic officer, chief student affairs officer, institutional 
research officer, and academic review and evaluation officer. Figure 25 presents the responses 
from associate of arts and all institutions. 

For associate of arts institutions, assessment managers responsible for day-to-day activities 
most often report to either the chief academic (43%) or chief executive (37%) officer. Hardly any 
report to the chief student affairs officer (13%), the institutional research officer (2%), or the 
academic review and evaluation officer ( 1 %). Compared to all institutions, assessment managers 
at associate of arts institutions are more likely to report to the chief executive officer and less likely 
to report to the chief academic officer. This finding parallels the greater centralized authority 
typically found in associate of arts institutions. 

Figure 25. Reporting Relationship of Individual with Operating Responsibility for Student 
Assessment Activities 
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Evaluating the Student Assessment Process 

The student assessment literature insists upon the importance of institutions regularly 
evaluating their assessment processes. Consequently, our survey inquired whether institutions 
have evaluated their student assessment plan or process and if so, what elements of their plan or 
process have been reviewed. 
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Status of Evaluation of Student Assessment Plan or Process 



Institutions were asked whether they have conducted a formal evaluation, an informal 
evaluation, are currently developing plans for an evaluation, or are not planning to evaluate their 
assessment process. The results for associate of arts and all institutions are depicted in Figure 26. 



Figure 26. Status of Student Assessment Evaluation 



« 

•I c 
| I 

to CL 
c a> 
“ GC 



100 

90 

80 

70 

60 

50 

40 

30 

20 

10 

0 






■ 












■ f--: 


• Y-i 




: *’ 1 '• »• -• 






.• : -fs ' ’!■’ 


. .. ■ . . 


■ AA 






; T >'• • r 










□ All 






- .. •- ,/e 














‘ 1 ■-.! v; v.:V. . 








' . ,. ;v . ... 




. V Y. •: 

•.VSijV-s ,? 




-C .a ; • ; 


• „... 
















formal 

evaluation 



informal 

evaluation 



planning an 
evaluation 



not 

evaluating 



Slightly more than half (52%) of all associate of arts institutions have evaluated their 
assessment plan. Of these, half have conducted a formal evaluation and half have conducted an 
informal evaluation. If they have not yet evaluated their assessment plan, associate of arts 
institutions are likely to be developing an evaluation plan (28%) and slightly less likely to not be 
planning to evaluate their assessment process (20%). Compared to all institutions, associate of arts 
institutions are slightly more likely to have conducted a formal evaluation and slightly less likely to 
have conducted an informal evaluation. 

Elements of Student Assessment Evaluation 

Those institutions that have formally or informally evaluated their student assessment process 
were asked which of eight elements were reviewed as part of the evaluation: student assessment 
plans and policies; structure and responsibility for student assessment; achievement of intended 
objectives for student assessment; reliability and validity of assessment instruments and methods; 
quality of data analysis; use of assessment information in institutional decision-making; problems 
encountered while conducting assessment; and the costs and benefits of student assessment. Results 
of this question for associate of arts and all institutions are shown in Figure 27. 

Among the associate of arts institutions that evaluate their student assessment process, most 
take several elements of that process into consideration. They are most likely to assess their 
student assessment plan and policies (79%), the achievement of intended objectives for student 
assessment (71%), the use of assessment information in decision-making (69%), and the problems 
encountered while conducting assessment activities (67%). These institutions are less likely to 
evaluate the quality of data analysis (49%), the reliability and validity .of assessment instruments 
and methods (58%), and the structure and responsibility for student assessment (61%). They are 
least likely to assess the costs and benefits of student assessment (23%). Compared to all 
institutions, associate of arts institutions are more likely to evaluate the reliability and validity of 
assessment instruments and methods, the use of assessment information in decision-making, the 
comparison of costs and benefits of student assessment, and the achievement of intended 
objectives for student assessment. 



O 

ERIC 



37 



45 



Figure 27. Elements of Student Assessment Evaluation 




Summary Observations 

The patterns described in this section indicate mixed institution-wide support for student 
assessment at associate of arts institutions. Although these institutions do not often mention 
student assessment in their mission statements and do not provide frequent institution-wide 
administrative or governing activities promoting student assessment, there are at least three 
indications of support. First, associate of arts institutions report a balance of external and internal 
purposes for conducting student assessment. This balance suggests that they are not solely 
responding to external mandates, but plan to make institutional improvements based on student 
assessment involvement and results. Second, respondents report that internal constituents are 
generally quite supportive of student assessment. Finally, most institutions have a planning 
committee for student assessment with broad representation of administrators and faculty. 

Despite the indications of support for student assessment on associate of arts campuses, 
our survey results also suggest that many of these assessment processes are administrative, rather 
than faculty, driven. Although both faculty and administrators are fairly supportive of student 
assessment, administrators are more supportive. Furthermore, student assessment plans on 
associate of arts campuses are more likely to be formal and centralized than decentralized. 
Academic affairs administrators not only are most likely to chair the planning group and to have 
approval authority for assessment changes, they also are likely to have operating responsibility for 
day-to-day student assessment activities. If not an academic affairs staff member, an institutional 
researcher is more likely to have day to day operating responsibilities than is a faculty member. 

The pattern of being more administratively than faculty driven is stronger for associate of 
aits institutions than it is for all institutions in our study. Associate of arts institutions are less 
likely to have a faculty governance committee addressing assessment and less likely to have a 
faculty member in charge of day-to-day responsibilities than are all institutions. Furthermore, at 
associate of arts institutions, the chief executive officer is more likely to have approval authority, 
the person in charge of day-to-day operating responsibilities is more likely to report to the chief 
executive officer, and the process itself is less likely to be decentralized than is the case in all 
institutions. Another difference between associate of arts and all institutions is that in associate of 
arts institutions student affairs staff and administrators appear to play a bigger role. They are more 
likely to be represented on student assessment committees and those who are in charge of day-to- 
day operating responsibilities are more likely to report to student affairs administrators than is the 
case in all institutions. A final difference is that associate of arts institutions are more likely to be 
engaging in student assessment to meet state mandates than are all institutions. 








6. ASSESSMENT MANAGEMENT PRACTICES AND POLICIES 



Reports on the existence of specific institutional practices and policies 
designed to promote student assessment management. Associate of arts 
institutions prefer unobtrusive practices and policies, such as distributing 
assessment results and providing access to student performance information 
as well as more obtrusive practices and policies such as using student 
assessment information in making either budget allocation or faculty 
evaluation and reward decisions. 

A fourth domain in our conceptual framework is that of assessment management practices 
and policies providing mechanisms for managing the student assessment process and directing the 
ways in which student assessment information is used throughout the institution. Assessment 
management practices and policies are suggested in the literature as powerful means through which 
institutions can support student assessment and encourage the use of collected assessment 
information. Conceptual dimensions of assessment management practices and policies such as 
their comprehensiveness, consistency, and the extent to which they are employed within an 
institution are expected to influence internal support for student assessment and the likelihood of 
achieving institutional impacts from assessment information. 

Our survey asked about specific institutional practices and policies promoting student 
assessment management. These items were factor analyzed to create nine comprehensive indices. 
In this section we will present results of these assessment management practice and policy indices, 
rather than of individual items. 

Practices for Managing Student Assessment 

What kinds of practices do institutions develop to manage and promote their student 
assessment process? We asked respondents about the existence of a variety of specific institutional 
practices to manage the student assessment process in four areas: academic resource allocation, 
student information systems, internal access to student assessment information, and distribution of 
assessment reports and studies. Scores for institutions for all specific practices in these four areas 
are presented in Appendix A. We used factor analysis to create indices of management practices in 
these four areas. Detailed information on these indices are provided in Appendix B Briefly, the 
four indices are: 

1. Academic Budget Decisions: whether the institution’s academic budget process compares 
academic units on student performance indicators and allocates resources competitively among 
them; or rewards them for improvement based on student performance indicators. 

2. Computer Support: whether the institution has a computerized student information system that 
includes student performance indicators; tracks students from application through graduation; and 
is integrated with faculty, curricular, and financial databases. 

3. Access to Student Information: whether assessment information on individual students is 
available to institutional researchers, assessment or evaluation professionals; senior academic 
administrators; department chairs or academic program administrators; student affairs 
professionals; and faculty advisors. 

4. Distribution of Reports: whether assessment reports are regularly distributed to students, 
faculty, academic administrators, student affairs professionals, employers, and the general public. 
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Figure 28 presents information on these four indices. Each was originally measured on a 
different scale. For the purposes of this graph, we have translated each index to a scale of 0-5. 
Therefore, we can determine which of these practices is more prevalent in associate of arts and all 
institutions in our study. 

Figure 28. Extent of Institutional Use of Assessment Management Practices 
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Scale: 0 = no practices in index used; 5 = all practices in index used. 

Of these four assessment management practices, associate of arts institutions are most 
likely to provide access on individual students’ performance to a number of internal constituents. 
For example, one of the items that comprises this index is whether student assessment data are 
accessible to institutional researchers — 83% of associate of arts respondents report that it is. If 
internal constituents have access to student assessment data, they should be able to make decisions 
based on these data, use the data to conduct studies, and even advise individual students based on 
their assessment information. 

The next most prevalent student assessment management practice is distributing student 
assessment reports to internal and external constituents. This index is comprised of whether 
assessment information is distributed to six individuals. Most associate of arts institutions 
regularly distribute reports to academic administrators (84%), faculty (69%), and student affairs 
professionals (67%). Hardly any associate of arts institutions regularly distribute reports to 
students (19%), the general public (10%), or employers (7%). 

Comparatively speaking, using computer information systems to manage the student 
assessment process is a less-often used practice. One of the items in this index is whether the 
institution has a student information system that tracks students from application through 
graduation. Less than half of associate of arts institutions (41%) report that they have such a 
system. Even fewer have either a system which includes student performance indicators or a 
system that is integrated with other institutional databases. These findings appear to contradict the 
literature that stresses that associate of arts institutions have advanced computerized systems to 
guide the student assessment process. -- --- - — — _ - - - 

Using the budget to compare and reward units or personnel is practically non-existent. For 
example only 3% of institutions report that they reward academic units for improvement based on 
student performance indicators. Compared to all institutions in our study, associate of arts 
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institutions are more likely to use all of the assessment management practices except for academic 
budget allocations. 

Policies Supporting and Promoting the Use of Student Assessment 

While the previous section focused on practices used to manage student assessment, this 
section focuses on the policies used to both support student assessment and guide how the 
resulting data are incorporated into other institutional processes. We asked survey respondents 
about the extent to which they have developed a variety of specific institutional policies to support 
and promote student assessment in four areas: student involvement in assessment; professional 
development for student assessment; training for student affairs staff; faculty evaluation and 
rewards; and academic planning and review processes. Factor analysis produced five indices of 
student assessment policies; detailed information on these indices can be found in Appendix B. 
Briefly, these five indices are: 

1. Student Involvement: extent to which the institution requires students to participate in student 
assessment activities, provides students with information on the purpose and uses of student 
assessment, and provides individual feedback regarding student performance results. 

2. Professional Development: extent to which the institution provides funds for faculty to attend 
assessment conferences, faculty workshops or consultative services on student assessment, 
assistance to faculty to improve their use of student assessment, and workshops/seminars for 
academic administrators on assessment. 

3. Student Affairs Training: extent to which the institution requires student affairs staff to receive 
training on assessment, and provides student assessment workshops for student affairs 
administrators. 

4. Faculty Evaluation: extent to which the institution considers evidence of student performance in 
faculty evaluation for promotion; incorporates evidence of student performance into faculty 
evaluation for salary and merit; considers faculty scholarship on assessment in promotion, tenure, 
or salary reviews; considers faculty participation in assessment in promotion, tenure, or salary 
reviews; and recognizes faculty for effective use of assessment. 

5. Academic Planning and Review: extent to which the institution incorporates student performance 
data into academic department or undergraduate program planning or review, general education or 
core curriculum review, course-level review and development, and review and planning for student 
academic support services. 

Figure 29 presents the means for these five indices for associate of arts and all institutions. 
Scores for the individual items that comprise these indices are in Appendix A. Of the five 
assessment management policies presented in Figure 29, associate of arts institutions are most 
likely to incorporate student assessment data'into academic planning and review processes. For 
example, respondents report that in some to many departments, student performance data are 
incorporated into undergraduate program planning or review— one of the items comprising this 
index. This finding provides evidence that associate of arts institutions are engaging in student 
assessment to improve undergraduate programs, a purpose they listed as moderate to very 
important. ______ 

Associate of arts institutions also make fairly extensive use of policies encouraging student 
involvement in assessment activities. For the three items that comprise this index, associate of arts 
institutions report that many departments require students to participate in assessment activities and 
some departments provide students with both information and feedback on student assessment. 
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Requiring participation and providing information on assessment purposes should increase student 
involvement, while providing individual feedback may improve student performance. 



Figure 29. Extent of Institutional Use of Assessment Management Policies 
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Associate of arts institutions are slightly more likely to offer training and workshops for 
student affairs staff than they are to offer professional development on student assessment to 
faculty and academic administrators. Actually, when comparing the individual items that comprise 
these two indices (see Appendix A), only a few departments offer assistance such as paid leaves, 
stipends, and course reductions to faculty to improve their use of student assessment. Since the 
mean for this item is low, it lowers the mean for the professional development index. Offering 
funds for faculty to attend assessment conferences and offering student assessment workshops to 
faculty and academic administrators is actually more prevalent than offering workshops on student 
assessment for student affairs administrators. 

Institutions are much less likely to evaluate and reward faculty based on student assessment 
participation or results. For example, respondents report that at their campus, less than a few 
departments consider faculty participation in assessment in promotion, tenure or salary reviews. 
Neither is student performance often used to reward or evaluate faculty. Associate of arts 
institutions responded that less than a few departments considered evidence of student performance 
in either promotion or salary and merit reviews. While these uses of student assessment data are 
fairly intrusive, one of the items in this index is more benign: publicly recognizing faculty for 
effective use of assessment. Nonetheless, associate of arts institutions using this strategy in only a 
few departments. 

In comparison with all institutions in our study, associate of arts institutions are more likely 
to use all of these assessment management practices and policies except for evaluating and 
rewarding faculty. On the one hand, faculty promotion decisions in associate of arts institutions 
are often based on seniority, thus the finding that student assessment data are not used in this 
process as much as it is in other institutions is to be expected. On the other hand, however, if 
associate of arts institutions proclaim that they are student-oriented, one might believe that student 
assessment data would be a driving force in making decisions on promoting faculty. 
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Summary Observations 



In section four we concluded that the assessment processes in associate of arts institutions 
appear to be administratively-driven. While the findings in this section do not contradict that 
conclusion, they do temper it somewhat. Administrators seem to be taking an unobtrusive 
approach in developing practices and setting policies. There is veiy little use of student assessment 
information in making either budget or faculty evaluation and reward decisions. Less obtrusive 
policies, such as distributing assessment results and providing access to student performance 
information are more prevalent. Associate of arts institutions are more likely to provide access to 
student assessment data than are all institutions. Also, compared to all institutions, associate of 
arts institutions are more likely to train student affairs staff on student assessment; this parallels the 
earlier finding that student affairs staff are more involved with student assessment at associate of 
arts institutions. 
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7. USES AND IMPACTS OF STUDENT ASSESSMENT 



Explores two important questions regarding how institutions use student 
assessment information in decision-making and the impact it has on 
institutions. . Associate of arts institutions are not using student assessment 
information to any great degree in making academic decisions and report that 
student assessment has had little impact on either their internal processes or 
their external relationships. 

As noted in our introduction, effective student assessment processes contribute to 
improvements in institutional and student performance. Consequently, two important questions in 
our research are: 1) how do institutions use student assessment information and 2) how does 
student assessment impact institutions? 

From the literature we identified and focused on two critical dimensions: the use of student 
assessment information in academic decision making, and the internal and external impacts on the 
institution that have resulted from student assessment. Our survey included ten academic decision 
and fourteen institutional impact items. Institutions’ scores for each use and impact item are 
presented in Appendix A and details on the indices are in Appendix B. We used factor analysis to 
create indices of uses and impacts in these five areas. Briefly, the five indices are: 

1 . Educational Decisions: the extent to which student assessment information is used in revising 
undergraduate academic mission or goals; designing or reorganizing academic programs or majors; 
designing or reorganizing student affairs units; allocating resources to academic units; modifying 
student assessment plans, policies, or processes; modifying general education curriculum; 
modifying student out-of-class learning experiences; creating or modifying distance learning 
initiatives; modifying teaching methods; and modifying student academic support services. 

2. Faculty Decisions: the extent to which student assessment information is used in decisions on 
faculty promotion and tenure; and salary increases or rewards. 

3. Faculty Impacts: whether student assessment has stimulated campus discussions of 
undergraduate education; contributed to faculty satisfaction; contributed to faculty interest in 
teaching; and led to changes in teaching methods used. 

4. Student Impacts: whether student assessment contributes to student satisfaction; affects student 
retention or graduation rates; affects student grade performance; and affects student achievement on 
external examinations. 

5. External Impacts: whether student assessment information affects student application or 
acceptance rates; allocation of state funding; evaluation from regional accreditation agency; private 
fund-raising results; success on grant applications; communications with external constituents; and 
institutional reputation or image. 

Figure 30 presents the means for these five indices for associate of arts and all institutions. 

_ Uses in Academic Decisions . ; _ 

Associate of arts institutions are not likely to use student assessment data in making 
academic decisions. They are more likely to make educational decisions (1.40) than they are to 
make faculty related decisions (1.20) using student assessment information. Within the 
educational decision index, associate of arts institutions are most likely to use student assessment 
information to modify student assessment plans, policies, or processes (2.70). While this practice 
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is the most prevalent of all the items comprising the educational decisions index, we know from 
section five that only half of all responding associate of arts institutions report that they have 
evaluated their assessment plan or process, either formally or informally. 



Figure 30. Institutional Uses and Impacts of Student Assessment Information 
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*Use scale: l=no action or influence unknown, 2=action taken, data not influential; 3=action taken, data somewhat 
influential; 4=action taken, data very influential. 

**Impact scale: l=not monitored, do not know; 2=monitored, negative impact; 3=monitored, no known impact; 
4=monitored, positive impact. 



Not only is the faculty related decision index mean lower than the educational decision 
index mean, but the two items comprising the faculty index have the lowest means of any item in 
both of these indices. Associate of arts institutions report that they are not at all likely to use 
student assessment information in making decisions on either faculty promotion and tenure (1.36) 
or faculty salary increases or rewards (1.30). These findings provide further evidence for the 
finding that associate of arts institutions are not likely to have policies integrating student 
assessment data into faculty evaluation and rewards procedures. 

Associate of arts institutions are not making great use of student assessment information in 
making either faculty or educational decisions. Compared to all institutions, they make even less 
use of student assessment information in faculty-related decisions, which parallels the finding that 
associate of arts institutions are less likely than all institutions to have policies integrating student 
assessment data into faculty evaluation and rewards procedures 

Institutional Impacts 

In terms of the impacts of student assessment information, associate of arts institutions are 
most likely to report that student assessment impacts students. Within this index, institutions are 
most likely to report that student assessment has impacted students’ retention or graduation rates 
(2.24), and least likely to report that student assessment has impacted students’ satisfaction (1.99). 
Many student assessment techniques require greater student effort and involvement, which may not 
lead to greater student satisfaction, especially in the short-run. 

Associate of arts institutions are less likely to report that student assessment has impacted 
faculty. Within this index, institutions are most likely to report that student assessment has led to 
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changes in teaching methods used (2.41) and least likely to report that student assessment has led 
to greater faculty satisfaction (1.71). Again, assessing students may take more faculty energy and 
time which could lead to decreased satisfaction, especially if faculty are in the early learning stages. 

Institutions were least likely to say that student assessment had impacted their external 
environment. The most commonly affected external domain is in the receipt of an evaluation from 
the regional accreditation agency (2.47). 

Overall, associate of arts institutions report that student assessment has not greatly impacted 
either their internally- or their externally-oriented processes and functions. Compared to other 
institutions, associate of arts institutions are slightly more likely to report that their students are 
impacted by student assessment and slightly less likely to report that either their faculty or their 
external environment is affected by student assessment. 

Summary Observations 

Our study has revealed that associate of arts institutions are not using student assessment 
information to any great degree in making academic decisions. The greatest use of these data is in 
evaluating student assessment processes and we know from section five that only half of associate 
of arts institutions have evaluated their student assessment plan or process. Neither are associate 
of arts institutions reporting that student assessment has impacted either their internal processes or 
their external environment. These findings indicate that student assessment has not yet become 
entrenched in associate of arts institutions. 
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8. KEY RELATIONSHIPS 



Internal influences, such as a mission emphasis on student assessment and 
excellence in undergraduate education; conducting assessment for internal 
improvement purposes; and providing a number of administrative and 
governance activities to support student assessment have more influence on 
associate of arts institutions’ involvement with student assessment than do 
external forces. Engaging in student assessment for internal purposes; 
collecting more student data; making use of non-traditional assessment 
methods; involving a great number of internal constituents through 
professional development offerings; distributing reports of assessment results; 
and conducting studies to link students’ performance to their interactions with 
the institution are related to more extensive uses of student assessment data in 
making academic decisions. Associate of arts institutions sponsoring more 
administrative and governance activities; using computer systems to support 
assessment; involving students; offering professional development 
opportunities to faculty and academic and student affairs staff; conducting 
further studies; and using the student assessment data they collect to evaluate 
faculty and plan and to review academic programs report that student 
assessment has had a greater impact on their internal processes and external 
relationships. 

In the previous sections we have examined the external influences on, the institutional 
approaches to, organizational and administrative support for, management practices and policies 
regarding, and the institutional uses and impacts of student assessment. Even more important is 
understanding how these domains are related to student assessment making a difference in 
institutional performance. That is, which external influences, institutional approaches to 
assessment, organizational and administrative support patterns, and assessment management 
practices and policies are most likely to promote the use of and positive impacts from student 
assessment information? 

In this section, we will examine three key relationships. First, we discuss how external 
influences affect an institution’s approach to student assessment (the type and extent of their use of 
student assessment). Then we examine how external influences and institutional characteristics, 
approaches, support patterns, practices and policies affect an institution’s use of student 
assessment data. Finally we examine how these domains lead to positive institutional impacts from 
student assessment. (The information in this section uses the indices discussed in previous 
sections. For detailed information on the indices, please refer to Appendix B.) 

External and Internal Influences on Student Assessment Approaches 

Are external forces more influential than internal forces in determining how associate of arts 
institutions approach student assessment? We used three multiple regression models to compare 
the influences of state characteristics related to student assessment, regional accreditation 
membership, and institution- wide support for student assessment on the extent of an institution’s 
use of three approaches to student assessment: cognitive assessment (the extent to which data are 
collected on measures of students’ cognitive performance); affective assessment (the extent to - 
which data are collected on measures of students’ affective development or performance); and post- 
college assessment (the extent to which data are collected on measures of students’ post-enrollment 
performance). In these three models, the predictor variables stem from three domains: state 
assessment approach, accrediting region, and institution-wide support. Since we were examining 
the influence of states’ assessment approaches, only public institutions were included in these 
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models. The predictor variables in this model are shown in Table 6 and defined in detail in 
Appendix B. 



Table 6. The Influence of Institution-Wide Support, State Assessment Approach, and 
Accrediting Region on the Extent of Institutional Approach to Student Assessment for Public 
Associate of Arts Institutions 







Extent 


of Institutional Approach 






Cognitive 


Affective 


Post-College 




Assessment 


Assessment 


Assessment 




Beta 


AR 2 


Beta AR 2 


Beta 


AR 2 


R 2 


• 


21 * * 


.12** 


.19 


* * 


Institution-Wide Support 












Mission Emphasis 


18** 


.04 


.13** .02 


.13** 


.02 


Administrative & Governance Activities 


13** 


.02 




.09* 


.01 


Administrator & Faculty Support 
Conduct for internal improvement 
Conduct for state 
Conduct for accreditation 


22** 


.09 


.22** .08 


.16** 


.04 


State Assessment Anoroach 












Authority Structure 

Form of State Assessment Initiative 

Common Indicators/Outcomes 












Accrediting Region 












Middle States 
North Central 








.14** 


.01 


New England 
Northwest*** 
Southern 


-.08* 


.01 


.09* .01 


.10* 


.01 


Western 


-.27** 


.07 


-.15** .02 


-.27** 


.09 



*p < .05; **p < .01 



***Since “accrediting region” was a categorical variable, Northwest Accrediting Region was left out of this 
regression because its affect on cognitive competencies, based on ANOVA, was closest to the mean. 



Cognitive Assessment 

Table 6 shows the regression results for the relationship of the predictor variables in the 
three domains on the extent of institutional use of cognitive assessment. This model is fairly 
strong, explaining 21% of the variance in the dependent measure. Which of the variables has the 
most influence on whether associate of arts institutions assess students’ cognitive abilities? Clearly 
the institution-wide patterns of support have more influence than the accrediting region, which in 
turn has more influence than the state. Within the institution-wide support domain, institutional 
emphasis on student assessment for internal improvement purposes is an important predictor of the 
extent of assessing students’ cognitive abilities. Emphasizing student assessment in the mission 
statement and the extent of the institution’s administrative and governance activities are also 
important. Finally, institutions in the Western and New England regional accrediting regions are 
less likely to assess students’ cognitive abilities. 
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Affective Assessment 



The second column in table 6 shows the regression results for the relationship of the 
predictor variables in the three domains to the extent of institutional use of affective assessment. 
This model is not as strong, explaining only 12% of the variance in the dependent measure. The 
extent of affective assessment conducted by associate of arts institutions is affected by both 
institution-wide support patterns and accrediting region. An internal purpose for conducting 
assessment and a mission emphasizing it are again both important influences. Institutions in the 
Western accrediting region are less likely to assess students’ affective skills or growth, while 
institutions in the Southern accrediting region are more likely to do so. 

Post-College Assessment 

Finally, Table 6 presents the regression results for the relationship of the predictor variables 
in the three domains on the extent of institutional use of post-college assessment. This model is 
stronger than the affective model and comparable to the cognitive model, explaining 19% of the 
variance in the dependent measure. The influences on the extent of data collected on post-college 
outcomes fall into the same pattern — institution-wide support patterns and accrediting region have 
the greatest impact. As was the case with cognitive assessment, internal purpose, administrative 
and governing activities, and academic mission are all predictors of the extent to which associate of 
arts institutions collect post-college data. Institutions in the Middle States and Southern regions are 
more likely to collect post-college data, while institutions in the Western region are again less likely 
to collect data. 

Institutional dynamics and accrediting region are the primary influences in all three of these 
student assessment approach models. Apparently, the drive for state-level accountability has not 
exceeded the influence of accreditation, and internal dynamics appear to be the driving force of all 
three approaches to student assessment. 

Influences on Using Assessment Information in Academic Decisions 

How do associate of arts institutions effectively promote and support the use of student 
assessment information in academic decision making? Within the domain of academic decisions, 
we created two indices reflecting the use of student assessment information: educational decision 
making and faculty decision making (these indices are described in detail in Appendix B). We 
used multiple regression to examine the influence of external forces, institutional size, institutional 
approach to student assessment, institution-wide support, and assessment management practices 
and policies on using student assessment information in both educational and faculty decisions. 

The predictor variables in this model are defined in detail in Appendix B. All institutions (public 
and private) were included in these models. Table 7 presents the results of these two regression 
models for associate of arts institutions. 

Influence on Use of Assessment Information in Educational Decisions 

The educational decisions model works well for associate of arts institutions, explaining 
4 1 % of the variance in the influence of student assessment data on educational decisions. While 
there are significant predictor variables from the domains of external influence, institutional 
approach, institution-wide support, and practices and policies, most of the significant variables are 
from either institutional approach or assessment management practices and policies. 

Within the domain of assessment management practices and policies, if institutions train 
student affairs staff in assessment, distribute assessment reports widely throughout their 
institution, have computer systems in support of assessment, use assessment data to evaluate 
faculty, and involve students in the assessment process, they are more likely to use student 
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Table 7. The Influence of External Influences, Institutional Context, Institutional Approach, 
Institution-Wide Support, and Management Practices and Policies on Using Student 
Assessment Information in Educational and Faculty Decisions for Associate of Arts 
Institutions 





Educational Decisions 


Faculty Decisions 




Beta 


AR? 


Bfita 


AR? 


Adjusted R 2 


.41** 


.12 


* * 


External Influences 








Middle States accrediting region 
North Central accrediting region 






-.11** 


.02 


New England accrediting region 
Southern accrediting region 
Western accrediting region 
State initiative for student assessment 


12** 


.02 






State approach to student assessment 
Accreditation influence 










Institutional Context 










Enrollment 








• ■ » 


Control (1 = private, 2 = public) 
Institutional ApproachJo Student Assessment 










Cognitive assessment 
Affective assessment 


.10* 


.09 






Post-college assessment 


.08* 


.01 






Number of instruments 










Student-centered methods 
External methods 


.13** 


.02 






Total assessment studies 

Institution-Wide Suoport for Student Assessment 


.16** 


.05 


.16** 


.03 


Mission emphasis 
Conduct for internal improvement 
Conduct for accreditation 
Conduct for state 


.12** 


.03 


-.09* 


.01 


Administrative & governance activities 
Administrator & faculty support 
Formal centralized policy 
Institution-wide planning group 
Conducted evaluation of assessment process 






-.09* 


.01 


Assessment Management Practices and Policies 










Academic Budget decisions 
Computer support 
Access to information 


.10** 


.01 






Distribution of reports 


.14** 


.02 






Student involvement 


.08* 


.01 






Professional development 
Student affairs training 


.16** 


.14 


.12** 


.01 


Faculty evaluation 1 


.10** 


.01 


n/inc 




Academic planning & review 2 


n/inc 




.15** 


.05 



'The factor “faculty evaluation” was not entered into the faculty decisions regression model, since many of the items 
comprising this factor were similar to items in the dependent factor. 

2 The factor “academic planning and review” was not entered into the educational decisions regression model, since 
many of the items comprising this factor were similar to items in the dependent factor. 
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assessment information to make educational decisions. Within the approach domain, the more 
studies associate of arts institutions conduct on the link between students’ performance and their 
educational experiences, the more student-centered methods institutions use in assessment, and the 
more cognitive and post-college data institutions collect, the more likely they are to use student 
assessment information to make educational decisions. 

External influences do not play a large role in determining whether an institution uses 
student assessment information to make educational decisions. However, associate of arts 
institutions in the Southern region are more likely than institutions in other regional accreditation 
regions to use student assessment information to make educational decisions. Neither is 
institution-wide support an important indicator of using student assessment information in making 
educational decisions. Only one variable from this domain is significant — if associate of arts 
institutions conduct assessment for internal improvement purposes, they are more likely to use 
student assessment information to make educational decisions. 

Influences on the Use of Assessment Information on Faculty Decisions 

The faculty decisions model does not work as well for associate of arts institutions, 
explaining only 1 1% of the variance. This result is not surprising, given that most associate of arts 
institutions do not use student assessment information to make faculty decisions. In our regression 
model, six predictor variables are significant. Two are from the assessment management practices 
and policies domain. If institutions both use assessment data in academic planning and review 
processes and offer professional development opportunities on student assessment, they are more 
likely to use student assessment data to make faculty-related decisions. There is one significant 
variable from the institutional approach domain: the more studies conducted by the institution on 
the link between students’ performance and their experiences with the institution, the more likely 
the institution is to use student assessment data to make faculty-related decisions. 

There are three significant predictor variables in this model that have a negative influence on 
whether institutions use student assessment data to make faculty-related decisions. Two of these 
variables are in the institution-wide support domain. If the institution has an institution-wide 
planning group for student assessment, they are less likely to use student assessment data to make 
faculty-related decisions. Similarly, if institutions purport that they conduct assessment to meet 
accreditation mandates, they are less likely to use student assessment data to make faculty-related 
decisions. Finally, our model demonstrates that institutions in the North Central region are less 
likely to use student assessment data to make faculty-related decisions. 

Influences of Student Assessment Information on Internal and External 

Institutional Performance 

How does the use of student assessment information affect various internal and external 
institutional performance dimensions? In our survey, associate of arts institutions reported 
whether, and the extent to which, student assessment has impacted various aspects of faculty and 
student performance and relationships with their external environment. We created three indices 
reflecting the impact of student assessment information on faculty, students, and the institutions’ 
external relations (these indices are described in detail in Appendix B and in section seven). We 
used multiple regression to examine the influences of external forces, institutional size, institutional 
. approach to_student assessment, institution-wide support, and assessment management practices - 
and policies on the impact of student assessment information on students, faculty, and the 
institution’s external environment. The predictor variables in this model are defined in detail in 
Appendix B. Table 8 presents the results of these three regression models for associate of arts 
institutions. 
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Table 8. The Influence of External Influences, Institutional Context, Institutional Approach, 
Institution-Wide Support, and Management Practices and Policies on Faculty, Student, and 
External Impacts for Associate of Arts Institutions 





Faculty Impacts 


Student Impacts 


External Impacts 




Beta 


AR 2 


Beta 


AR 2 


Beta 


AR? 


Adjusted R 2 > 


.28 


♦ * 


.22** 


.23 


** 


External Influences 














Middle States accrediting region 
North Central accrediting region 
New England accrediting region 
Southern accrediting region 
Western accrediting region 
State initiative for student assessment 
State approach to student assessment 
Accreditation influence 


-.09* 


.01 










Institutional Context 














Enrollment 














Control (1 = public, 2 = private) 
Institutional Accroach to Student Assessment 














Cognitive assessment 
Affective assessment 
Post-college assessment 
Number of instruments 
Student-centered methods 
External methods 


.09* 


.01 






.09* 


.01 


Total assessment studies 
Institution-Wide Succort for Student 

Assessment 


.14** 


.06 


.14** 


.03 


.10* 


.01 


Mission emphasis 

Conduct for internal improvement 

Conduct for accreditation 


-.12** 


.01 










Conduct for state 

Administrative & governance activities 
Administrator & faculty support 


.17** 


.04 






.12** 


.03 


Formal centralized policy 
Institution-wide planning group 
Conducted evaluation of assessment process 


.08* 


.01 










Assessment Management Practices and Policies 














Academic Budget decisions 










O 

OO 

* 


.01 


Computer support 
Access to information 


. .09* 


.01 


.15** 


.04 


.18** 


.05 


Distribution of reports 
Student involvement 
Professional development 
Student affairs training 


.17** 


.11 


.13** 


.02 


.10* 


.01 


Faculty evaluation 


.11* 


.01 


, .11** 


.01 


.13** 


.02 


Academic planning & review . 


.10* 


_:02 


.21**-- 


.13 


.13** 


.10 



*p < .05; **p < .01 
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Influences on the Impact of Student Assessment Information on Faculty 

What variables predict how extensive an impact student assessment will have on faculty 
and their activities? The first two columns of Table 8 show the statistically significant predictors of 
faculty impacts The model on faculty impacts is fairly strong, predicting 28% of the variance in the 
dependent measure. Institution-wide support for student assessment is key, according to the results 
of our regression model. If associate of arts institutions offer administrative and governance 
activities on student assessment and evaluate their assessment process, it is more likely that then- 
assessment activities will impact faculty. Conversely, if associate of arts institutions 
conduct assessment to meet accreditation requirements, it is less likely that their assessment 
activities will impact faculty. 

The assessment management practices and policies used by the institution are also 
important determinants of whether their assessment activities will affect faculty. Institutions that 
offer professional development, evaluate faculty using assessment information, use student 
assessment information in their academic planning and review process, and have computer systems 
to support assessment activities are more likely to report that their assessment activities have 
impacted faculty. How an institution approaches student assessment is also a key predictor in this 
model. Associate of arts institutions that conduct studies linking students’ performance to their 
interactions with the institution and institutions that make use of student-centered methods of 
student assessment are likely to report that their assessment activities have impacted faculty. 
External influences are less important in this model. However, if an institution is located in a state 
that has a centralized approach to student assessment, meaning that it requires common indicators 
and outcomes, there is less likely to be an impact from engaging in student assessment on that 
institution’s faculty. 

Influences on the Impact of Assessment Information on Students 

The middle two columns of Table 8 show which variables predict how extensive an impact 
student assessment will have on students. The model on student impacts is slightly weaker than 
the model on faculty impacts, predicting 21% of the variance in the dependent measure The 
assessment management practices and policies used by an associate of arts institution appear to 
have the greatest influence on whether student assessment activities impact student performance. 
Institutions that use student assessment data in their academic planning and review process, have 
computer systems supporting, student assessment, train student affairs staff in assessment, and 
evaluate faculty based on assessment information are likely to report that their assessment activities 
have impacted students. In terms of how institutions approach assessment, the more studies they 
conduct on linking students’ performance to their interactions with the institution, the more likely 
student assessment is to impact students. 

Influences on the Impact of Assessment Information on External Institutional Relations 

Finally, the last two columns of Table 8 show which variables predict how extensive an 
impact student assessment will have on an institution’s external relations. The model on external 
impacts is practically equal in strength to that of student impacts, predicting 23% of the variance in 
the dependent measure. All of the significant predictor variables in this model are internal variables. 
The strongest variables are from the assessment management practices and policies domain. 
Institutions that have computer systems to support assessment initiatives, use student assessment 
information in their academic planning and review processes, use student assessment data in 
evaluating faculty, involve students in the assessment process, and use student assessment data in 
making budget decisions are more likely to report that engaging in student assessment has had an 
external impact. Within the domain of institutional approach, the more studies institutions conduct 
on linking student performance to their institutional experiences and the more instruments 
institutions use to collect student data, the more likely the institutions are to report that engaging in 
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student assessment has had an external impact. Finally, within the domain of institution-wide 
support, institutions that offer more administrative and governance activities on student assessment 
are more likely to report that engaging in student assessment has impacted their external 
environment. 



Summary Observations 

In examining how internal and external forces compare in influencing associate of arts 
institutions’ approaches to assessing students, it is clear that internal influences are much stronger. 
While different accrediting regions spur varying levels of engagement in collecting student 
assessment data, state policies and structures do not appear to be strong influences on the extent to 
which associate of arts institutions collect student assessment data. Three internal strategies are 
particularly important in determining the extent of student data collected: emphasizing student 
assessment and excellence in undergraduate education in the mission statement, conducting 
assessment for internal improvement purposes, and providing a number of administrative and 
governance activities on student assessment. 

If associate of arts institutions are to benefit from engaging in student assessment, they 
should not only be collecting student data, but also using these data to make academic decisions. 
Although institutions in different accrediting regions vary in terms of how much they use 
assessment information in making academic decisions, external influences in general are extremely 
limited. Institutions that use student assessment data in making academic decisions commit to their 
assessment process — evidenced by the strategy of engaging in student assessment for internal 
purposes, rather than to meet accrediting requirements, and by the existence of computer systems 
in support of assessment. These institutions also collect more student data and to make use of non- 
traditional methods of assessing students. They also involve a great number of internal 
constituents through their professional development offerings, including student affairs staff and 
students themselves. Finally, they go beyond merely collecting student assessment data to 
distributing reports of assessment results and conducting studies on the link between students’ 
performance and their interactions with the institution. 

If engaging in student assessment makes a difference in associate of arts institutions, they 
should report that it is impacting their internal processes and external relations. External influences 
were not important predictors of student assessment impacts with one exception: if the institution 
conducted assessment to meet accreditation requirements, they were not likely to report that their 
assessment activities had an impact. Institutions at which the assessment process has had an 
impact have committed support to the assessment process through administrative and governance 
activities and through creating computer systems to support the assessment process. They also use 
diverse and extensive collection methods — both traditional and non-traditional. They involve 
students in the assessment process and they offer professional development opportunities on 
student assessment to faculty and student and academic affairs staff. Finally, they go beyond 
merely collecting assessment data to both conducting further studies and to using the data they 
collect. They use these data to evaluate faculty, plan and review academic programs, and make 
budget decisions. 
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9. EXECUTIVE SUMMARY: RESEARCH RESULTS AND RECOMMENDATIONS 



Recommendations for associate of arts institutions based on the results of our 
research are outlined in this section. This study and monograph highlight the 
importance of viewing student assessment from a critical institutional 
perspective and underscore three important realities: 1) for student 

assessment to enhance student, faculty, academic, and institutional 
performance, it has to be viewed as an institutional process, not just a series 
of student assessment activities; 2) a great deal of organizational, 
administrative, and academic activity is or can be invested in initiating, 
managing, and using student assessment; and 3) that a systematic look at 
those institutional activities can enhance an institution ’s ability to plan for and 
use student assessment effectively. 

Student Assessment: A Critical Institutional Perspective 

The intent of our survey and of this monograph is to highlight the importance of viewing 
student assessment from a critical institutional perspective and to underscore three important 
realities: 1) for student assessment to enhance student, faculty, academic* and institutional 
performance, it has to be viewed as more than just a series of student assessment activities; 2) a 
great deal of organizational, administrative, and academic activity is or can be invested in initiating, 
managing, and using student assessment; and 3) a systematic look at those institutional activities 
can enhance an institution’s ability to use student assessment effectively. 

The data reported in this survey provide a national profile of what associate of arts 
institutions are currently doing to support and promote the use of student assessment. This 
evidence provides insight into the activities institutions are doing extensively, those which merit 
greater attention, and those which make a difference in improving institutional performance. We 
begin with a summary of our research findings and recommendations concerning specific 
assessment activities conducted within associate of arts institutions. Then we discuss how 
associate of arts institutions can use the Institutional Support for Student Assessment (ISS A) 
inventory and the Framework for Institutional Support for Student Assessment (Figure 1 of this 
monograph) as a guide for examining their student assessment process and functions. We 
conclude by suggesting that the results of the national survey used in conjunction with institutional 
self-examination can serve as the basis for redesigning or planning a student assessment process 
which can enhance an institution’s academic performance. 

Student Assessment in Associate of Arts Institutions: Results and 

Recommendations from the National Survey 

Institutions that have committed resources to assessing the development of their 
undergraduate student assessment activities should be able to use the student assessment data they 
collect in organizational decision-making and to document impacts from their assessment activities. 
Associate of arts institutions that are engaging in undergraduate student assessment but do not feel 
that they are profiting from the process as much as they could be may want to adopt the strategies 
of institutions that have reported using and being positively impacted by assessment information. 
Our research has demonstrated that most associate of arts institutions have made only limited use of 
student assessment data and given little attention to monitoring impacts from assessment. 

However, they have adopted a wide variety of student assessment approach measures, institution- 
wide activities supporting student assessment, and assessment management practices and policies. 
Our research shows that external influences do play some role in encouraging institutions to 
conduct and use student assessment, but there is much that institutions themselves can do to 
initiate, support, and benefit from their undergraduate student assessment efforts. Our research 
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identified strategies from the domains of student assessment approach, institution-wide support for 
student assessment, and assessment management practices and policies that lead to greater 
institutional uses and impacts from undergraduate student assessment information. In the 
following sections, we highlight the student assessment activities that associate of arts institutions 
are currently engaging in to a great extent, the activities that they may want to augment, and the 
activities that are critical to enhancing the use of student assessment data in academic decisions 
leading to positive institutional impacts. 

Student Assessment Uses and Impacts 

If institutions are benefiting from their student assessment process, they should report that 
they are using resulting assessment data in academic decisions and that the use of these data has a 
positive impact on both internal processes and external relationships. In this section, we describe 
the extent to which associate of arts institutions are using student assessment data and the extent to 
which these data have had an impact on the institution. Suggestions for increasing institutional 
uses and impacts of student assessment data are outlined in the following sections. 

Uses of Student Assessment Information . Most associate of arts institutions tire not using 
student assessment data to make academic decisions. Our study did not attempt to discern why 
institutions are not making more extensive use of these data. Perhaps institutional decision-makers 
do not have sufficient data to inform decisions concerning institutional policies and practices. This 
may be particularly so if assessment data collection focuses on student inputs (e.g., basic college- 
readiness skills or academic intentions of entering students) or outputs (e.g., graduation or 
employment rates of exiting students) without attempting to measure changes in student 
performance or development, or without examining the relationship between student performance 
and development and facets of their experiences within the institution (e.g., course-taking patterns, 
financial aid, advising policies). Assessment data use may also be limited if institutions have not 
created a formal mechanism for incorporating this data into decision-making processes. Associate 
of arts institutions should examine why they are not making more use of student assessment data. 
For example, is the type of data being collected not useful for informing decisions? Are there 
concerns about data quality? Is assessment data not easily accessible or not widely distributed to 
decision makers? Raising questions of this nature will assist institutions in understanding how 
they may increase the use of assessment information in decision making. 

Impacts of Student Assessment Information . Similarly, associate of arts institutions are 
reporting very minimal impacts on students, faculty, or their external relationships from student 
assessment data. For the most part, we found that associate of arts institutions are simply not 
monitoring whether student assessment information has had an impact. Such monitoring is 
important if institutions are to evaluate the institutional benefits of engaging in student assessment. 
The remainder of this report suggests institutional practices, policies, and strategies that may 
promote institutional uses and impacts of student assessment data. 

External influences 

External influences, such as state and accreditation requirements, can be strong motivators 
for institutions to engage in student assessment. In addition, a variety of postsecondary 
organizations and external funding sources offer support for institutions’ assessment programs. 
This section describes these influences on associate of arts institutions and provides, suggestions 
for deriving the most benefit from these external relationships. 

State Influences . Most of the associate of arts institutions that responded to our survey are 
public institutions and are under the purview of a state mandate for student assessment. Most of 
these institutions responded that they were involved in developing the details of the state mandate. 
However, some institutions were not involved in this development. Where such opportunities 
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exist, institutions should try to be involved with planning at the state level in order to craft 
mandates that are fair, practical, and beneficial to the institutions. Institutions located in states 
where the assessment mandate requires common indicators and outcomes are less likely to report 
that student assessment has impacted their faculty. Perhaps in these states, the mandates have 
omitted a link between student assessment and faculty impacts. If associate of arts institutions 
desire that their faculty be impacted by their student assessment process yet fall under a state 
mandate that requires common indicators and outcomes for all public institutions (or even all 
associate of arts institutions) in the state, they may want to work with the state to craft indicators 
that are more relevant for their individual institutions and faculty. 

Accreditation Influences . Just as most associate of arts institutions are subject to a state 
mandate for student assessment, most have completed a regional accreditation review requiring 
undergraduate student assessment. In fact, for most associate of arts institutions, accreditation 
requirements were either an important reason to initiate student assessment or an incentive to 
increase involvement in assessing students. Accreditation agencies apparently have a strong 
influence on associate of arts institutions’ student assessment activities. Some regional 
accreditation associations have more influence on institutions’ student assessment activities than do 
others. For example, institutions in the Western accrediting region collect less student assessment 
data than do institutions in the Southern accrediting region. Institutions should be cognizant of the 
influence of their accrediting region. While it is important to respond to accreditation requirements, 
our research shows that if responding to accreditation requirements is the major purpose for 
engaging in student assessment activities, student assessment information is less likely to be used 
in making faculty-related decisions or to impact faculty. Perhaps when assessment is conducted 
mainly to satisfy external requirements, it is less likely to affect those in the classroom. If associate 
of arts institutions want their assessment information to affect faculty, they should be quite sure 
that they are engaging in student assessment for internal, as well as external, purposes. 

Other External Influences . In addition to state mandates and accreditation requirements, 
there are other externally-oriented considerations bearing on associate of arts institutions’ student 
assessment activities. Many associate of arts institutions have made use of conferences and 
publications on student assessment provided by professional and regional accreditation 
associations. Although a variety of external agencies and programs offer grants to improve or 
support institutions’ assessment practices, the majority of associate of arts institutions have not 
received such grants. In addition, all institutions in our study are more likely to receive FIPSE and 
private foundation or corporate grants than are associate of arts institutions. Associate of arts 
institutions interested in augmenting their assessment efforts may want to explore these 
opportunities for external support. 

Institutional Approach to Student Assessment 

Institutional approach to student assessment refers to institutional decisions regarding the 
collection and analysis of student assessment information. Dimensions along which assessment 
approaches can be differentiated include: the type and extent of student assessment data collected; 
the methods used to collect assessment data; and the analyses conducted and reported for collected 
data. Associate of arts institutions have adopted a variety of approaches to collecting and analyzing 
student assessment data. This section describes these approaches and makes suggestions for 
adopting assessment approaches that will lead to uses and impacts of student assessment 
information. _ _ 



Type and Extent . Most associate of arts institutions emphasize the collection of certain 
types of student assessment data including basic college-readiness skills, academic intentions, 
academic progress, satisfaction, and post-college vocational and educational outcomes. While 
collecting these data is important, they reflect a measurement of student placement, student 
satisfaction, and movement through the educational process, rather than cognitive or affective 
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development. Associate of arts institutions collect less extensive information on their students’ 
higher order skills, general education competence, competence in the major, vocational skills, 
personal and affective development, and involvement with the institution. Collecting these types of 
data, particularly at more than one point in time during students’ involvement with the institution, 
would provide associate of arts institutions with more substantive data concerning students’ 
cognitive and affective growth. Not only will collecting such data provide a richer understanding 
of student growth, but our research has shown that the more data institutions collect, the more 
likely they are to use this assessment data in making educational decisions. Associate of arts 
institutions should review the type and extent of assessment data being collected and, where 
needed, broaden and deepen these collection efforts. 

Assessment Methods . When collecting student assessment data, associate of arts 
institutions as a whole tend to use tests and other written instruments. Some units or departments 
within associate of arts institutions make use of less traditional assessment methods such as 
observations of student performance; student portfolios or comprehensive projects; student 
performance in capstone courses; student interviews or focus groups; employer interviews or focus 
groups; alumni interviews or focus groups; and interviews with withdrawing students. Associate 
of arts institutions should increase the use of these alternative measures of collecting student 
assessment data in order to gather data that is not accessible via more traditional tests and surveys. 
Such alternative measures also tend to involve faculty more in the measurement process through 
participation in designing and administering these methods, and interpreting the data collected. Our 
research found that the more extensive the data collection methods an institution employed, in 
terms of using a greater number of instruments and tests and more student-centered methods, the 
more likely the institution is to use the data to make educational decisions and to report that the data 
has impacted both faculty and external relationships. Associate of arts institutions should 
encourage or support extensive and alternative data collection methods within their units if they 
want the resulting student assessment data to be useful and to have a positive impact on their 
institution. 

Assessment Studies . Institutions in our study reported whether they examine aspects of the 
institution that affect students’ performance. Slightly more than half of associate of arts institutions 
do conduct such studies. The most frequently conducted studies examine student financial aid 
and/or concurrent employment; admissions standards or policies; student course-taking patterns; 
exposure to different instructional or teaching methods; and academic advising patterns. These 
studies are important if institutional decision-makers are to understand how their decisions affect 
students’ performance. Our research has found that institutions that conduct more studies are more 
likely to report that they use student assessment information in making both educational and 
faculty-related decisions and that this assessment information has impacted their students, faculty, 
and external relationships. Clearly, conducting such studies is extremely important and there is 
room for associate of arts institutions to carry out more of them. 

Institution-Wide Assessment Support Strategy 

Associate of arts institutions have developed a variety of strategies to support their student 
assessment processes such as including student assessment in the academic mission statement; 
developing internal purposes for engaging in assessment; sponsoring institution-wide activities to 
promote involvement in and support for assessment; adopting an institutional plan or policy for 
student assessment; establishing processes and structures for planning and coordinating ... 
assessment; and evaluating the assessment process. This section describes the strategies associate 
of arts institutions use to support student assessment and provides suggestions regarding specific 
strategies that enhance the likelihood of achieving institutional benefits from student assessment 
information. 
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Mission Emphasis . One way to determine the degree of institutional support for an activity 
is to examine whether the activity is emphasized in the mission statement. While most associate of 
arts institutions emphasize excellence in undergraduate education, only half describe intended 
outcomes for student assessment and only about one-fifth reference student assessment. If student 
assessment is indeed a core value of the institution, this value should be communicated via the 
mission statement. Our research found that institutions that emphasize assessment in their mission 
statements are more likely to collect a greater extent of student assessment data. 

Assessment Purposes. Similarly, support for assessment can be detected by understanding 
an institution’s purpose for engaging in student assessment. Associate of arts institutions report 
that accreditation mandates and improving undergraduate education are both very important 
purposes for engaging in assessment. This finding is encouraging, as it indicates that institutions 
both are aware of external demands and are purposefully engaging in student assessment to 
improve internal processes. Associate of arts institutions should continue to maintain such a 
balance, especially as our research found that institutions that conduct assessment for internal 
improvement purposes collect more assessment data and report that they use this data to make 
educational decisions. 

Institution-Wide Activities . Another measure of support for student assessment is the 
number of institution-wide administrative and governance activities used to promote assessment. 
Approximately half of associate of arts respondents report that they offer annual forums on student 
assessment; provide regular workshops for academic and student affairs administrators; and have a 
faculty governance committee that regularly addresses assessment issues. While these efforts are a 
good start, they are only adopted by half of our respondents. Furthermore, our results show that 
associate of arts institutions seldom provide incentives or rewards to administrators or academic 
units for engaging in or using the results of student assessment. Associate of arts institutions 
could increase the use of such regular activities if they want to demonstrate institution-wide support 
for assessing students. Not only will offering such activities demonstrate support, but our 
research found that institutions that offer more administrative and governance activities collect more 
student assessment data and report that these data have an impact on both faculty and their external 
relationships. 

Administrative and Faculty Support . Our survey also asked respondents to describe their 
opinion of the level of support for student assessment given by various internal constituents. Only 
academic affairs administrators were described as very supportive. Does this finding indicate a 
feeling of ambivalence toward assessing students? Even the chief executive officer was described, 
on average, as being only somewhat supportive of student assessment. If the chief executive 
officer is not very supportive of student assessment, there are bound to be difficulties in promoting 
it, funding it, and rewarding people for engaging in it. Furthermore, if faculty governance 
committees are only somewhat supportive of student assessment, faculty members may not fully 
engage in assessing students. Periodically measuring degrees of internal support for student 
assessment may be a useful means for associate of arts institutions to gauge the effectiveness of 
efforts to promote assessment. 

Institutional Plans and Policies for Assessment . The types of plans and policies institutions 
develop for assessing students are telling of their support for student assessment. Virtually all 
responding associate of arts institutions have some type of plan or policy for assessing students. 
Half of associate of arts institutions have developed a formal plan that specifies assessment 
activities for all academic programs. The next most popular form is a formal guidance policy that 
identifies institution-wide'activities to be conducted by a central office or committee. Apparently 
associate of arts institutions favor centralized plans and policies. The fact that they have plans and 
policies is encouraging as it indicates that they have devoted time and attention to the student 
assessment process. 
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Planning and Coordinating Assessment . Furthermore, most associate of arts institutions 
have some type of planning group for student assessment. Typically, an academic affairs 
administrator is responsible for chairing these groups. In addition to academic administrators, 
most of these groups are staffed by faculty, student affairs administrators, and institutional 
researchers. This broad representation of constituents is a sign that there is wide input on student 
assessment issues. However, only one-third of these planning groups has a student 
representative. Associate of arts institutions may want to consider involving students in these 
groups. Student representation increases the likelihood of understanding how students feel about 
student assessment. Once institutions understand how students view assessment, they can attempt 
to overcome student objections to participating in assessment activities. As associate of arts 
institutions consider including students in the assessment planning and coordination process, they 
should be aware that our research found that institutions that have a planning group are less likely 
to use student information to make faculty decisions. 

Evaluation of Assessment Process . Whether an institution has evaluated its assessment 
plan or policy is often an indication of the importance it accords student assessment as an 
institutional activity. Half of associate of arts institutions have evaluated their plan or policy for 
student assessment. All associate of arts institutions should consider evaluating their plans and 
policies in order to better understand whether their plan is meeting its objectives and whether the 
effort expended to assess students is benefiting the institution. Furthermore, those institutions that 
have evaluated their assessment process report that their faculty have been impacted by student 
assessment data. 

Assessment Management Practices 

Assessment management practices refer to specific practices intentionally devised by 
institutions to manage their student assessment efforts. Four specific areas of practice were 
identified as potential influences on student assessment: academic resource allocation, student 
information systems, internal access to student information, and distribution of assessment reports 
and studies. This section describes the practices used by associate of arts institutions and provides 
suggestions for using these practices to maximize uses and impacts of student assessment 
information. 

Academic Budget . Associate of arts institutions are not likely to allocate budgetary 
resources based on assessment engagement or results. If this practice was augmented, institutional 
leaders could demonstrate that they consider student assessment to be a high priority. In addition, 
our research found that institutions that do allocate resources based on assessment engagement or 
results are more likely to report that their student assessment information has a positive impact on 
their external relations. Perhaps external founders are stipulating that institutions use assessment 
results to hold individuals and departments accountable. 

Computer Support . Currently, fewer than half of associate of arts institutions report that 
they have a student information system that tracks students from application through graduation. 
Even fewer have either a system which includes student performance indicators or an integrated 
database. Along with improving the management of student assessment data, creating relational 
databases with student assessment information that can be linked to other organizational data 
should facilitate studying the link between students’ performance and their institutional 
interactions. Our research found that institutions benefit greatly from having computerized systems 
to support and manage the assessment process. These institutions report that they use their student 
assessment information to make educational decisions and that this information has had a positive 
impact on faculty, students, and their external relationships. Associate of arts institutions should 
increase their use of such systems; doing so is clearly a very important endeavor. 
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Access to Assessment Information . Providing broad access to student assessment 
information is important if institutions want their constituents to make use of the information. 

Most associate of arts institutions do provide broad internal access to student assessment 
information. Providing such access makes it easier for internal personnel to intervene with or give 
feedback to students regarding their performance. 

Assessment Report Distribution . Not only is providing access to information helpful in 
making use of assessment information, but distributing reports of studies done using student 
assessment data is also important. The more people are aware of student assessment activities and 
results, the more likely they will be to use the information in their own decision making. While 
most associate of arts institutions regularly distribute reports to traditional internal constituents such 
as faculty, academic administrators, and student affairs professionals, hardly any associate of arts 
institutions regularly distribute reports to students, the general public, or employers. Associate of 
arts institutions should increase the number of constituents who receive their reports. Our research 
found that the greater the number of constituents who receive student assessment reports, the more 
likely the institution is to use student assessment data to make educational decisions. 

Assessment Management Policies 

Assessment management policies refer to institutional policies devised to both support 
student assessment and to direct the use of student assessment information. Five content 
dimensions of assessment management policies were examined: student involvement in 
assessment; professional development on assessment for faculty and academic administrators; 
training in assessment for student affairs personnel; faculty evaluation and rewards; and academic 
planning and review processes. This section describes the use of these policies by associate of arts 
institutions and provides suggestions regarding policies that maximize institutional uses and 
impacts of student assessment information. - 

Student Involvement . While associate of arts institutions report that many of their 
departments require students to participate in assessment activities, they also report that only some 
departments provide students with both information about and feedback on their assessments. 
Explaining the importance of assessment and providing individual feedback to students should not 
only lead to improved uses and impacts of assessment information, but increase students’ 
participation in assessment activities while improving their performance. Our research found that 
the more the institution involves students in the assessment process, the more likely the institution 
is to use assessment data to make educational decisions and the more likely this data is to impact 
the institution’s external relationships. 

Professional Development . Associate of arts institutions do offer professional development 
on student assessment to their faculty and academic administrators. Their professional 
development policies, however, are more likely to involve activities that do not significantly affect 
time in the classroom (for example, funds to attend assessment conferences or workshops on 
assessment) than those that involve greater expenditure of administrative and faculty time (for 
example, paid leaves or course reduction to develop or improve assessment practices). While 
understandable from a fiscal perspective, this emphasis on providing professional development 
support of shorter duration may not provide faculty with adequate time to learn about or develop 
new assessment techniques. Offering a variety of professional development opportunities should 
increase the level of faculty and administrative involvement in and support for student assessment. .. .. 
Our research found that those institutions that offer extensive professional development are more 
likely to use their student assessment information to make decisions regarding faculty and this 
information is more likely to have an impact on their faculty. Associate of arts institutions should 
review and consider increasing the array of professional development opportunities on student 
assessment that are offered to their academic personnel. 
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Student Affairs Training . When designing professional development opportunities, 
associate of arts institutions should not neglect their student affairs personnel. Involving student 
affairs staff in student assessment may lead to greater student involvement as the staff have 
opportunities for promoting assessment activities while they work with students. Currently, 
associate of arts institutions are including student affairs personnel in their professional 
development initiatives — a policy decision that should benefit these institutions. Our research 
found that those institutions that provide student assessment training to their student affairs 
personnel are more likely to use assessment information in making educational decisions and to 
report that this assessment information has had an impact on their students. 

Faculty Rewards and Evaluation . Associate of arts institutions rarely have policies that 
include assessment-related criteria in faculty evaluation and rewards. Although the wisdom of this 
linkage has been the cause of much debate in the assessment literature, our research found that 
associate of arts institutions that do include assessment-related criteria in their faculty evaluation 
and reward processes are more likely to use assessment information to make educational decisions 
and to report that this information has impacted their students, faculty, and external relationships. 
Institutions may understandably be reluctant to tie faculty evaluation and rewards to assessment 
results or indicators of student performance, but there are many other options to consider. For 
example, institutions can include criteria concerning faculty scholarship in assessment, evidence of 
using assessment to improve teaching and learning, or participating in assessment decision-making 
in reviewing and evaluating faculty. 

Academic Planning and Review . Another use for student assessment information is in 
planning and reviewing academic programs and courses. Associate of arts institutions currently 
make fairly extensive use of student assessment information in their academic planning and review 
processes, particularly in reviewing academic programs. Our research found that using student 
assessment information in academic planning and review is instrumental to gaining positive 
impacts from the assessment process. The more institutions use assessment information in 
academic planning and review, the more likely they are to use assessment information to make 
faculty decisions and the more likely they are to report that this assessment information positively 
impacts faculty, students, and their external relationships. 

Inventorying the Institutional Student Assessment Process 

Regardless of an institution’s history of and support for student assessment, it is important 
to take stock of what the institution is currently doing. The Institutional Framework presented in 
section two (Figure 1 of this monograph) provides a comprehensive perspective for such an 
examination. The Institutional Support for Student Assessment (ISSA) inventory included as 
Appendix A provides a useful quasi-objective instrument for identifying the specific dimensions 
and activities associated with the student assessment process. 

Institutions are encouraged to identify a team of faculty and administrators most 
knowledgeable about and involved with student assessment to examine their institution’s activities 
on all the dimensions in the ISSA inventory. While most items are objective (e.g., which types of 
measures are being used and how extensively), the actual pattern on campus may not be widely 
known. Other items are more subjective (e.g., purposes for conducting student assessment) and 
can provide the basis for good discussion. . 

This inventory of the institutional student assessment process then can be compared with 
national data for associate of arts institutions which were presented in Appendix A and summarized 
in the previous sections of this monograph. This inventorying and comparison process may 
highlight activities, policies, and practices not currently used; identify areas needing greater 
attention; or focus on inconsistencies in patterns of activity (e.g. inconsistencies between stated 
purposes for student assessment and actual uses of student assessment data in academic decisions). 
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For institutions with an extensive history of involvement with student assessment, such an 
inventory may serve as a useful basis for identifying new activities to be undertaken, for improving 
their existing activities and processes, for identifying issues or controversies that have been 
avoided, or for redesigning processes that may not be worth the current expenditure of effort and 
resources. For institutions with less experience with student assessment, the inventory may 
identify existing activities on which to build or help focus attention in the institution on the 
importance of student assessment in improving institutional performance. In either type of 
institution, the inventory and self evaluation process should help both to focus faculty and 
administrative attention on the importance of viewing student assessment as an institutional process 
and to deal with it more systematically — linking the various domains of activity with institutional 
improvement. 



Planning for Student Assessment 

Student assessment in most higher education institutions has emerged, often sporadically, 
over the past decade due to the need to respond to an accreditation self study, a new state mandate, 
an academic administrator who promoted it, a faculty group who embraced it for their unit, or an 
institutional researcher or program review officer who was engaged in studies of student 
performance. While, according to the data in this report, some institutions are beginning to 
develop a plan or policy for student assessment or create a group responsible for it, there is little 
evidence of systematic planning that links the student assessment approach to external demands 
and to internal institutional governance and management patterns; develops the organizational and 
administrative activities, practices, and policies to support it; and then uses the data for academic 
decisions and monitors their impact. Clearly there are significant institutional differences and 
complex issues to be addressed if student assessment is to have positive effects. These deserve 
some systematic, planned attention. The results of an institutional inventory can provide the basis 
for such an effort. 

While we do not advocate a cookbook or standardized approach to planning for student 
assessment (each institution needs to design its own planning approach to reflect its own 
governance and leadership styles and traditions), the Institutional Framework (Figure 1 in section 
two) and the institutional inventory provide a useful basis for redesigning or planning a student 
assessment process. Using that self-assessment, the following are planning issues that need to be 
addressed: 

External Influences 

1 . What is the nature of our state assessment process? How is it formulated and what are 
its requirements/implications for us? 

2 . What are the accreditation requirements for our institutional and key professional 
accreditation bodies? 

3 . What do some of our primary external constituents expect of our graduates? 

4 . What are sources of support (educational, financial, technical) for our student 
assessment efforts? 

Institution-Wide Support Patterns 

1 . What emphasis is placed on student assessment in our mission statements? What are the 
intended purposes of our institution’s assessment efforts? 
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2. What institution-wide administrative, governance, and academic activities support 
student assessment? Who are key leadership support groups and how are they 
involved? 

3 . What is the nature of our institution-wide plan or policy for student assessment? The 
role and membership of a coordinating or planning body? And the pattern of authority 
and responsibility for administering the process? 

4. Is the student assessment process to be evaluated? By whom? Using what criteria? 

Approaches to Student Assessment 

1 . Should we have an institution-wide or decentralized (by academic unit) approach to 
student assessment? 

2. What types of measures are to be used? How widely? At what points in time? 

3 . What types of instruments and methods are appropriate? What technical or professional 
support does that require? 

4. What studies of the influence of students’ educationally related experiences on their 
performance are to be done? What reports of student performance are to be prepared 
and distributed? 

Assessment Management Practices and Policies 

1 . What assessment management practices exist to guide student assessment? ( i.e., 
resource allocation, information systems, data access, and report distribution practices.) 

2. What institutional policies promote the use of student assessment? (i.e., policies on 
student involvement, professional development, student affairs training, faculty 
evaluation, and academic planning.) 

Uses and Impacts of Student Assessment 

1 . How will we assure use of student assessment information on educationally- and 
faculty-related academic decisions? 

2. How will we monitor the impact of student assessment on our students, faculty, 
academic and instructional patterns, and on our external relationships? 

While planning for student assessment may not resolve all of these questions, they should 
be addressed as should issues Of the balance between the effort and resources required to maintain 
the institution’s student assessment process and the educational and institutional benefits. When 
planning is combined with inventorying, these two processes can become a powerful mechanism 
for understanding and improving institutional student assessment endeavors. Inventorying 
existing student assessment processes is a first step toward understanding the nature and extent of 
an institution’s student assessment activities.^ Jhis. can be followed by a planning_process that 
considers the recommendations for associate of arts institutions that were presented in this section. 
Using a systematic planning approach should increase the likelihood that student assessment will 
contribute to improved institutional performance. 
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Inventory of 
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For The Research Program on 
Institutional Support for Student Assessment 




NCPI 

1 National 
Center for 
Postsecondary 
Improvement 



NCPI - Project 5.2 
University of Michigan 
Ann Arbor, Michigan 48 1 09- 1259 
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An Introduction to the ISSA 



The Institutional Support for Student Assessment Inventory (ISSA) was developed as part of a national research 
program examining the Organizational and Administrative Support for Student Assessment for the National Center 
for Postsecondary Improvement (NCPI). The ISSA is designed as an institutional inventory of the organizational 
and administrative practices that have been designed and implemented to support the use of student assessment on 
your campus. 

Institutional Support Practices are those organized activities, policies, and procedures that your institution has 
intentionally designed to enhance the practice of student assessment. Student Assessment refers to those activities 
focused on measuring dimensions of student performance other than traditional end of course grading. 

This national survey is designed to identify institutional support practices for undergraduate student assessment. The 
project also examines the factors influencing the adoption of various support practices and how those practices 
enhance the impact of student assessment for institutional improvement. 

We understand that being selected for this survey will require a commitment of time to complete and we appreciate 
your involvement. This instrument is also intended as an institutional self-assessment inventory to facilitate 
examination of your institution’s own organizational and administrative practices which support student assessment. 
We encourage each institution to use the survey in this manner. You will receive a summary report of survey 
responses to all compare with your own institutional profile. # 

N Completing the ISSA 

The main purpose is to obtain a profile of your institution’s current approach to undergraduate student assessment 
and its support practices. The inventory may be completed by one individual or group of individuals who are most 
familiar with the patterns of undergraduate student assessment on your campus. It should take less than one hour to 
complete. 

• Please keep in mind that the questions refer to undergraduate education at your institution. 

• Respond to each item in the questionnaire to the best of your knowledge. 

The questionnaire is coded to allow follow up only. Individual institutions will not be identified in any analyses or 
reports. 

Return the completed questionnaire in the enclosed return envelope. Any questions concerning the survey can be 
addressed to the following: 

National Center for Postsecondary Improvement Project 5.2 
School of Education " 

University of Michigan 
610 E. University, Room 2339 
Ann Arbor, MI 48109-1259 
Phone: 734-647-2464 
Fax: 734-936-2741 
Email: ncpi.proj52@umich.edu 

Marvin W. Peterson, Project Director _ 



NCPI is funded by the U.S. Department of Education’s Office of Educational Research and Improvement under OERI 
grant number R309A60001 



©1997, The Regents of the University of Michigan 
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I. Institutional Approach to Student Assessment 



A . Type, Extent and Timing of Student Assessment 

We are interested in your institution’s routine practices of collecting different types of undergraduate student 
performance data, the extent to which they are collected, and when they are collected. For each of the following 
content types of undergraduate student performance data: 

1) indicate the extent to which each type is collected 

2) each type of data collected, check whether it is collected at entry, during enrollment, at exit, or a 
combination of these data collection points. 



lXB£ 



Currently Enrolled Students 



1 . Student academic intentions or 
expectations 

2. Basic college-readiness 
skills (reading, writing, 
mathematics, etc.) 

3. Higher-order skills (critical 
thinking, problem solving) 

4. General education competencies 

5. Competence in major field 
of study (discipline- or 
program-specific knowledge) 

6. Vocational or professional skills 

7. Personal growth and 
affective development 
(values, attitudes, social 
development, etc.) 

8. Student experiences and 
involvement with institution 

9. Student satisfaction with 
institution 

10. Student academic progress 
(retention, graduation rates) 

Former Students 

1 1. Vocational or professional 
outcomes (career goals, job 
attainment or performance) 

12. Further education 
(transfer, degree attainment, 

- - - graduate study) - 

13. Civic or social roles 
(political, social or 
community involvement) 

14. Satisfaction and experiences 
with institution after leaving 





Extent 






Timing 






Collected 


Collected 


Collected 


Collected 


Collected 


Collected 


Not 


for some 


for many 


for all 


at 


while 


at 


Collected students 


students 


students 


entry 


enrolled 


exit 




(circle one number for each item) 


(check all that apply for each item) 


1 


2 


3 


4 










AA 


All 












3.38 


3.25 












3.56 


3.44 












1.88 


2.10 




— 


— 


— 




2 . 41 


2.55 




— 


— 


— 




2.30* 


2.60 












2.25 


2.11 







__ _ _ 


— 




1.77* 


2.12 












2.35 


2.57 












2.86 


2.96 






. 


— 



3.58 3.76 



2.73 2.72 



--- 2.71 - 2.69 — 



1.37* 


1.80 


2.57 


2.63 
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B. Student Assessment Instruments 



Does your institution employ institutionally or externally developed instruments or tests for the following types of 
undergraduate student assessment information? (circle all that apply for each item): 

Source of Instrument 



Content of Instrument 


Not 

used 

1 


Institutionally 

developed 

2 


State 

provided 

3 


Commercially 

available 

4 


1 . Student plans, goals, or expectations 


AA: 18% 


60% 


8% 


24% 




All: 21% 


51% 


4% 


32% 


2. Basic college-readiness skills 


AA: 1% 


21 %* 


14% 


85 %* 


(reading, writing, mathematics, etc.) 


All: 6% 


38% 


11% 


67% 


3. Higher-order skills 


AA: 51% 


26% 


2% 


27% 


(critical thinking, problem solving) 


All: 42% 


29% 


2% 


32% 


4. General education competencies 


AA: 35% 


38% 


5% 


33% 




All: 32% 


40% 


5% 


33% 


5. Competence in major field of study 


AA: 30% * 


55% 


14% 


26 %* 


(discipline- or program-specific knowledge) 


All: 19% 


64% 


12% 


39% 


6. Vocational or professional skills (excluding 


AA: 29% 


50% 


14% 


25% 


licensure exams) 


All: 34% 


43% 


14% 


25% 


7. Personal growth and affective development 


AA: 54 %* 


29% * 


2% 


21% 


(values, attitudes, social development, etc.) 


All: 38% 


39% 


2% 


30% 


8. Student effort, experiences or involvement 


AA: 29% 


57% 


5% 


17% 


with institution 


All: 21% 


60% 


4% 


24% 


9. Student satisfaction with institution 


AA: 4% 


75% 


12% 


28% 




All: 4% 


73% 


8% 


35% 


10. Alumni satisfaction and experiences 


AA: 16% 


73% 


12% 


9% 




All: 10% 


78% 


8% 


15% 
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C . Other Student Assessment Methods 

To what extent does your institution use the following methods to collect undergraduate student assessment 



information? (circle one number for each item): 








Other Student Assessment Methods No * Used *P . 

used some units 

1 2 
AA 

1. Observations of student performance 

(simulations, demonstrations, lab) 2.22 


Used in 
most units 
3 

All 

2.26 


Used in 
all units 
4 


2. Student portfolios or comprehensive projects 


1.95 


2.10 




3. Student performance in capstone courses 


1.78* 


2.15 




4. Student interviews or focus groups 


1.65 


1.84 




5. Transcript analysis 


2.19 


2.16 




6. External examination of students 
(licensure exams, external reviewers) 


2.01 


2.02 




7. Special surveys of or interviews with 
withdrawing students 


2.26 


2.40 




8. Alumni interviews or focus groups 


1.80 


1.90 




9. Employer interviews or focus groups 


1.98 


1.87 





D . Student Sub-Popuiations 

Does your institution use different assessment methods for the following sub-populations of undergraduate students? 
(check for each item): 







Different 








AA 


All 


l. 


Adult students 


3% 


10% 


2. 


Part-time students 


3% 


5% 


3. 


Minority students 


2% 


2% 


4. 


Distance education students 


21% 


22% 



Same as Other Students 



* “Unit” refers to academic areas such as departments, divisions, schools, or colleges. 
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E. Student Assessment Studies 



Does your institution conduct studies of the relationship between the following experiences and students’ 
performance (check Ml that apply): 





A A 


All 


1. Student course-taking patterns 


26% 


26% 


2. Exposure to different instructional or teaching methods 


25% 


21% 


3. Patterns of student-faculty interaction 


11% 


14% 


4. Extra-curricular activities 


15% . 


24% 


5. Residence arrangements 


6% * 


21% 


6. Student financial aid and/or concurrent employment 


27% 


30% 


7. Admission standards or policies 


27%* 


42% 


8. Academic advising patterns 


24% 


26% 


9. Classroom, library and/or computing resources 


19% 


17% 


10. Do not study the relationship between the above experiences and student performance 45% 


38% 



F. Student Performance Profiles or Reports 

Does your institution provide profiles or reports of appropriate student performance information at the following 
levels of aggregation (check MI that apply): 



• • , *: 


A A 


All 


1. Institution wide 


66% 


69% 


2. Schools or colleges 


13 %* 


31% 


3. Academic programs or departments 


61 % 


65% 


4. Special populations or subgroups/students 


46% 


46% 


5. By course or groups of courses 


46 %* 


36% 


6. Do not provide any reports 


13% 


11% 



II. Institutional Support for Student Assessment 
A. Institutional Emphasis 

1 . Your institutional mission statement explicitly (check all that apply): 



A A All 

a. emphasizes excellence in undergraduate education as an institutional priority 75% 82% 

b. identifies the educational outcomes intended for your students 50% 52% 

c. refers to student assessment as an important institutional activity 21% 19% 

d. does not explicitly mention any of the above 14% 11% 



2. For how many years has your institution engaged in student assessment? 




71 

79 



B. 



Purpose of Student Assessment 



The following are often intended purposes of an institution’s undergraduate student assessment process. Please rate 
the importance of each for your institution, (circle one number for each item): 



Purpose 


No 

Importance 

1 


Minor 

Importance 

2 


Moderate 

Importance 

3 






AA 


All 


1. Preparing institutional self-study 








for accreditation 




3.61 


3.86 


2. Meeting state reporting 
requirements 

3. Guiding internal resource 




3.37* 


2.89 


allocation decisions 
4. Guiding undergraduate academic 




2.83 


2.71 


program improvement 




3.38 


3.43 


5. Improving the achievement of 
undergraduate students 

6. Improving faculty instructional 




3.50 


3.48 


performance 




3.06 


3.02 



7. Other (briefly describe): 



Very 

Important 

4 



C . Administrative and Governance Activities 



Institutions have introduced a variety of administrative or governance activities that address or promote student 
assessment. Does your institution engage in any of the following activities? (check ah that apply): 



1 . 

2. 

3. 

4. 

5. 

6. 

7. 



Annual presidential or other institution-wide initiatives, forums 
or seminars on assessment 

Rewards or incentives for academic and student affairs administrators who 
promote use of student assessment in their unit 
Incentives for academic units to use student assessment information 
in their evaluation and improvement efforts 

Student assessment workshops for academic and student affairs administrators 
Board of trustees committee that addresses student assessment 
Faculty governance committee that addresses student assessment issues 
Student representation on student assessment committees 



AA 


All 


48% 


41% 



4% 


6% 


24% 


27% 


.57% 


56% 


10% 


13% 


50% 


58% 


28% 


33% 
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D. Support for Student Assessment 

Use the scale below to rate the degree to which various groups within your institution support undergraduate student 
assessment activities (circle on£ number for each item): 



Very 

Unsupportive 

1 


Somewhat 

Unsupportive 

2 

AA 


Neutral, 

Unknown 

3 

ALL 


Supportive 

4 


Very 

Supportive 

5 


Board of trustees 


3.87 


3.84 






Chief executive officer 


4.47 


4.41 






Academic affairs administrators 


4.64 


4.64 






Student affairs administrators 


4.38 


4.33 






Faculty governance 


3.87 


3.80 






Students 


3.40 


3.33 






. Planning and Coordinating Student 


Assessment 









1 . Which of the following best describes your institution’s plan or policy for undergraduate student 
assessment? Your institution (check all that apply): 



a. 



b. 



c. 



d. 

e. 

f. 
g- 



has a formally adopted institutional plan or policy requiring specified 
undergraduate student assessment activities of all academic units 


AA 


All 


or programs 

has a formally adopted plan or policy for undergraduate student assessment 
in some academic units or program areas (e.g. general education 


54% 


50% 


or academic majors) 

has a formally adopted institutional plan or policy requiring all academic 
units or programs to develop their own undergraduate student 


20% 


19% 


assessment plan 

has a formally adopted institutional plan or policy stipulating institution- 


24 %* 


39% 


wide activities to be conducted by a central committee, office, or officer 
has no formal plan or policy but academic units or programs are encouraged 


38% 


38% 


to conduct their own undergraduate student assessment activities 
is currently developing a plan or policy for undergraduate student 


10% 


13% 


assessment 

does not have an undergraduate student assessment plan or policy 


14% 


17% 


(SKIP TO E-6) 


4% 


4% 



2. Is there an institution-wide group (committee, task force, etc.) that is primarily responsible for ongoing 
planning and policy setting for undergraduate student assessment? (check one ): 



a. yes 

b. no (skip to Question E-5) 



A A 
73% 



All 

70% 



3. If yes, who serves on this group? (check all that apply): 







A A 


All 


a. 


Chief executive officer 


17% 


13% 


b. 


Academic affairs administrator(s)/staff 


86% 


86% 


c. 


Student affairs administrator(s)/staff 


67 %* 


54% 


d 


Institutional research administrator(s)/staff 


67% 


61% 


e. 


Academic review and evaluation administrator(s)/staff 


24% 


24% 


f. 


Student assessment administrator(s)/staff 


37% 


32% 


g- 


Faculty 


91% 


91% 


h. 


Students 


28% 


33% 


i. 


Other 


13% 


12% 



Who has executive responsibility for or who chairs the institution-wide group responsible for the ongoing 
planning or policy-setting process for undergraduate student assessment? (check all that apply): 







AA 


All 


a. 


Academic affairs administrator 


54% 


55% 


b. 


Student affairs administrator 


13% 


7% 


c. 


Institutional research officer 


18% 


18% 


d 


Academic review and evaluation officer 


3% 


5% 


e. 


Student assessment officer (if separate) 


8% 


8% 


f. 


Faculty member 


30% 


31% 


g- 


Other 


12% 


11% 



5. Who approves any changes in your institution’s plan or policies for undergraduate student assessment? 
(check ah that apply): 







AA 


All 


a. 


Board of trustees 


24% 


17% 


b. 


Chief executive officer 


57 %* 


45% 


c. 


Chief academic affairs officer 


72% 


75% 


d. 


Chief student affairs officer 


28% 


20% 


e. 


Institutional research officer 


20% 


18% 


f. 


Academic review and evaluation officer 


7% 


8% 


g- 


Student assessment officer 


12% 


10% 


j- 


Student government 


0.4% 


1% 


h. 


Academic senate or other faculty committee(s) 


30% 


39% 


i. 


Faculty union 


3% 


4% 


k. 


Other 


16% 


14% 
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6. Who has operational responsibility for your institution’s day-to-day undergraduate student assessment 

activities (e.g., instrument development, data collection, analysis, and reporting)? (check aH that apply): 







AA 


All 


a. 


Academic affairs administrator 


42% 


45% 


b. 


Student affairs administrator 


24% 


20% 


c. 


Institutional research officer 


49% 


45% 


d 


Academic review and evaluation officer 


6% 


9% 


e. 


Student assessment officer 


18% 


15% 


f. 


Faculty member(s) 


27% 


33% 


g. 


Other 


11% 


13% 


h. 


No one (SKIP TO QUESTION E8) 


4% 


3% 



7. 


To whom does the individual with operational responsibility for day-to-day student assessment activities 




directly report? (check one): 












AA 


All 




a. 


Chief executive officer 


37% 


29% 




b. 


Chief academic officer 


43% * 


56% 




c. 


Chief student affairs officer 


13% 


7% 




d. 


Institutional research officer 


2% 


3% 




e. 


Academic review and evaluation officer 


1% 


2% 




f. 


Other 


11% 


10% 



8. 


Is there an office which provides faculty consultation in using student assessment for instructional 






improvement or curriculum development? (check one): 


AA 


All 




a. yes 


47% 


47% 




b. no 


53% 


53% 


9. 


If ves. what is the name of the office? 







F. 


Evaluating Your Institution’s Student Assessment Plan or Process 




1 . 


Has your institution evaluated its undergraduate student assessment process? 


(check one): 










AA 


All 




a. 


yes, with a formal evaluation 


26% 


22% 




b. 


yes, with an informal evaluation 


26% 


27% 




c. 


currently developing evaluation plans (SKIP TO SECTION HI) 


28% 


29% 




d. 


not currently evaluating or planning to evaluate assessment 
process (SKIP TO SECTION 


20% 


21% 
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2. In evaluating your institution’s student assessment process, which of the following elements of that 
process were reviewed? (check aU that apply): 



a. 


your student assessment plan and policies 


AA 

79% 


All 

81% 


b. 


the structure and responsibility for student assessment 


61% 


64% 


c. 


achievement of your institution’s intended objectives for 
student assessment 


71% 


70% 


d. 


reliability and validity of student assessment instruments and 
methods 


58% 


54% 


e. 


quality of data analysis 


49% 


51% 


f. 


use of student assessment information in institutional 
decision-making 


69% 


66% 


g- 


the problems encountered while conducting student assessment 
activities 


67% 


69% 


h. 


comparison of the costs and benefits of student assessment 


23% 


22% 



III. External Influences on Institutional Student Assessment Activities 



State Role (FOR STATE-FUNDED institutions only; all others skip to question III. B-l) 




Was your state’s plan/requirement for student assessment primarily developed (check onell: 








AA 


All 


a. 


by state-level officials 


17% 


16% 


b. 


through joint consultation between state officials and institutional 
representatives 


45% 


39% 


c. 


no statewide plan or requirement for student assessment exists 
(SKIP TO III. B-l) 


38% 


46% 


State 


requirements for student assessment (check aU that apply): 


AA 


All 


a. 


were an important reason for your institution to initiate undergraduate 
student assessment 


48% 


45% 


b. 


have increased your institution’s involvement in undergraduate student 








assessment 


69% 


62% 


c. 


have not been a factor in your institution’s undergraduate student 
assessment activities 


15% 


22% 


d. 


have been a negative influence on your institution’s undergraduate 
student assessment activities 


4% 


4% 
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3. Your state’s reporting requirements include (check all that apply): 







AA 


All 


a. 


evidence that a student assessment plan is in place 


69% 


68% 


b. 


measurement of state-mandated student performance indicators 


66% 


64% 


c. 


institutionally-devised student performance indicators 


48% 


49% 


d. 


evidence of institutional use of student assessment information 


53% 


52% 



4. How has your state higher education agency reviewed or evaluated your institution’s undergraduate student 
assessment plan or process after it was implemented? (check aft that apply): 







AA 


All 


a. 


reviewed by state officials 


46% 


42% 


b. 


reviewed using external reviewers 


20% 


16% 


c. 


required an institutional self-review 


28% 


24% 


d. 


no post hoc review has occurred (skip to Question B-l) 


36% 


44% 


The state review of your institution’s undergraduate student assessment plan 


or process 


included (check all 



that apply): 







AA 


All 


a. 


review of your institution’s student assessment process itself 


48 %* 


67% 


b. 


comparison of your institution’s student performance record with your 
past performance 


41% 


44% 


c. 


comparison of your institution’s student performance record with peer 
institutions 


38% 


36% 


d. 


comparison of your institution’s student performance record with 
institutions in your state 


36% 


38% 


e. 


other (briefly describe) 


10% 


10% 


Regional 


Accrediting Role in Student Assessment 




Has your institution gone through a regional self study accreditation review which required undergraduate 


student assessment? (check one): 










AA 


All 


a. 


yes 


80% 


80% 


b. 


no 


20% 


29% 



2. Regional accreditation agency requirements for undergraduate student assessment (check Ml that apply): 







AA 


All 


a. 


were an important reason for your institution to initiate undergraduate student 
assessment 


62% 


64% 


b. 


have increased your institution’s involvement in undergraduate student 








assessment 


75% 


79% 


c. 


have not been a factor in your institution’s undergraduate student assessment 
activities 


15% 


12% 


d. 


have had a negative influence on your institution’s undergraduate std. 
assessment activities 


0.8% 


0.9% 
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3. 



Your institution’s regional accreditation agency requires (check Ml that apply): 







AA 


All 


a. 


evidence that a student assessment plan or process is in place 


89% 


90% 


b. 


intended institutional uses of student assessment information 


72% 


73% 


c. 


results of student assessment 


66% 


66% 


d. 


evidence of actual institutional use of student assessment information 


78% 


77% 


e. 


unfamiliar with regional accreditation requirements for student assessment 


4% 


5% 



C. External Sources of Support for Assessment 

1 . Has your institution received external grants to improve undergraduate student assessment practices from any of 
the following? (check Ml that apply): 







AA 


All 


a. 


FIPSE 


2% 


6% 


b. 


other federal agencies (please identify): 


10% 


7% 


c. 


a state incentive program 


8% 


7% 


d. 


private foundations or corporate sources (please identify): 


2% 


6% 


e. 


no known external grants received 


81% 


79% 



2. Has your institution used any of the following student assessment services offered by the following 
postsecondary organizations? (check Mi services that apply for each type<)f organization): 



Student Assessment Service Used 



Type of Postsecondary Organization 

a. Professional associations (Institutional, 
disciplinary, or administrative) 


Not used or 
not available 

AA: 35% 
All: 29% 


Consultation 

services 

11% 

13% 


Assessment 

conferences 

43% , 

51% 


Training 

workshops 

30% 

32% 


Publications 
or research 
reports 

46% 

51% 


b. Regional accrediting association 


AA: 35% 
All: 30% 


20% 

19% 


39% 

41% 


29% 

32% 


41% 

45% 


c. State-level agency 


AA:42%* 
All: 54% 


19% 

14% 


33% 

26% 


31% 

22% 


26% 

22% 


d. Consortium of institutions 


AA: 56% 
All: 53% 


12% 

13% 


25% 

30% 


20% 

18% 


16% 

20% 
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IV* Academic Management Policies and Practices for Student Assessment 



Institutions have a wide array of formally organized policies, activities, and procedures intended to enhance or support 
the collection and use of undergraduate student assessment information. The following policies and practices have 
been identified in many institutions. 

For Questions a through D, indicate whether the following policies or practices exist at your 

INSTITUTION. 



A. Resource Allocation for Student Assessment (check all that apply): 





AA 


All 


1. An explicit operating budget allocation is made to support student assessment. 


47% 


49% 


2. An academic budget process that considers student performance indicators 
in resource allocation to academic units. 


26% 


23% 


3. An academic budget process that compares academic units on student performance 
indicators and allocates resources competitively. 


2% 


2% 


4. An academic budget process that rewards academic units for improvement based on 
their own past student performance indicators. 


3% 


3% 


B. Student Assessment Information System (check aU that apply): 


AA 


All 


1. Key student assessment activities have been scheduled into the academic calendar. 


58% 


57% 


2. A computerized student information system which includes student performance 
indicators. 


34% 


28% 


3. Student information system tracks individual students from application through 
graduation. 


41 % 


42% 


4. Student assessment database integrated with faculty, curricular, and financial 
databases. 


14% 


10% 



C. Access to Individual Student Assessment Information (check all that apply): 



Student assessment information on individual students is available to: 





AA 


All 


1. Institutional research, assessment or evaluation professionals 


83% 


76% 


2. Senior academic administrators 


77% 


72% 


3. Department chairs or academic program administrators 


78% 


73% 


4. Student affairs professionals 


70%* 


58% 


5. Faculty advisors 


71% 


66% 
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D. Distribution of Student Assessment Reports and Studies (check aM that apply): 



Student assessment reports and studies or appropriate summaries are regularly distributed to: 





AA 


All 


1. Students 


19% 


19% 


2. Faculty 


69% 


67% 


3. Academic administrators 


84% 


86% 


4. Student affairs professionals 


67% 


58% 


5. Employers 


7% 


5% 


6. The general public 


9% 


8% 



For Questions E through H, use the following scale to indicate the extent to which each of the 
FOLLOWING POLICIES AND practices exist At your institution (Circle on£ number for each item). 





Not 


Done in 


Done in 


Done in 


Done 




done 


a few 


some 


many 


in 


E. Student Policies on Student Assessment 


at all 


depts. 


depts. 


depts. 


most 




1 


2 


3 


4 


depts. 

5 






AA 




All 




1 . Students are required to participate in student assessment 












activities 




3.81 




3.77 




2. Incentives are provided to encourage students to 
participate in student assessment activities 

3. Information regarding the purpose and uses of student 




1.72 




1.87 




assessment is provided to students 
4. Students are provided with individual feedback regarding 




3.49 




3.52 




their own student performance results 




3.38 




3.21 




F. Professional Development 




AA 




All 




1 . Faculty are required to learn about or receive training on 












student assessment 




2.76 




2.47 




2. Funds for faculty to attend or present at professional 
conferences on student assessment are available 

3. Workshops, seminars, or consultative services for 




3.41* 




3.08 




faculty on the use of student assessment in course 
design or instruction are offered 




3.09 




2.90 




4. Assistance for faculty in the form of paid leaves. 












stipends, mini grants or course reduction to improve use 
of student assessment is provided 




2.12 




2.00 




5. Workshops and seminars for department chairs, deans. 












and other academic administrators to improve use of 
student assessment in their unit is provided 




- 2.76 




2.55 




6. Student affairs staff are required to learn libout or receive 
training related to student assessment 

7. Student assessment workshops for student affairs 




2.51 




2.22 




administrators are provided 




2.54* 




2.22 
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G. Faculty Evaluation and Rewards 



AA 



All 



1. Faculty evaluation for promotion considers evidence of 
student performance in their classes (not just student 



teaching evaluation) 


1.69 


1.84 


2. Faculty evaluation for annual salary and merit increases 
incorporates evidence of student performance 


1.41 


1.56 


3. Faculty scholarship on or innovative uses of student 
assessment is considered in promotion, tenure, or salary 
reviews 


1.74 


2.01 


4. Faculty willingness to use or to participate in student 
assessment activities is considered in faculty promotion, 
tenure, or salary reviews 


1.85 


1.99 


5. Faculty receive public recognition or awards for 
innovative or effective use of student assessment 


1.62 


1.58 


6. Faculty hiring process considers experience or skill in 
student assessment 


1.84 


1.68 


7. Faculty are encouraged to assess student learning in their 
classes 


4.18 


3.99 
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Not 


Done in 


Done in 


Done in 


Done 




done 


a few 


some 


many 


in 


H. Academic Planning and Review 


at all 


depts. 


depts. 


depts. 


most 


1 


2 


3 


4 


depts. 

<; 


Your institution incorporates student performance data into 
the following processes: 




AA 




All 


J 


1. Academic department or undergraduate program planning 












or review 




3.65 




3.67 




2. General education or core curriculum review 




3.61 




3.55 




3. Course-level review and development 

4. Review and planning for student academic support 




3.57 




3.36 




services 




3.22 




3.09 





V. Impacts of Student Assessment 
A. Decision Making 

To what extent has the use of information available from your undergraduate student assessment process influenced 
the following actions? (circle qq£ number for each item): 



Institutional Actions 



1 . Revising your undergraduate academic mission or 
goals 

2. Designing or reorganizing academic programs or 
majors 

3. Designing or reorganizing student affairs units 

4. Allocating resources to academic units 

5. Modifying student assessment plans, policies, or 
processes 

6. Deciding faculty promotion and tenure 

7. Deciding faculty salary increases or rewards 
(release time, travel funds, etc.) 

8. Revising or modifying general education 
curriculum 

9. Creating or modifying student out- of-class 
learning experiences (e.g. internships, service 

learning) _ 

10. Creating or modifying distance learning 
initiatives 

11. Modifying instructional or teaching methods 

12. Modifying student academic support services 

(e.g. advising, tutoring) 



O 




No action or 


Action taken. 


Action taken. 


Action taken. 


influence 


data not 


data somewhat 


data very 


unknown 


influential 


influential 


influential 


' 1 


2 


3 


4 




AA 


All 






2.06 


2.06 






2.46 


2.54 






1.88 


1.91 






1.88 


1.81 






2.70 


2.61 






1.36 


1.46 






1.30 


1.39 






2.39 


2.47 






2.00- 


2.14 






1.88 


1.72 






2.51 


2.47 






2.56 


2.56 
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B. Institutional Impacts 



Have you monitored the following institutional indicators and been able to document the impact of student 
assessment information on them? (circle one number for each item): 





Not monitored. 


Monitored, 


Monitored, 


Monitored, 


Internal Impacts 


do not 


negative 


no known 


positive 


know 


impact 


impact 


impact 




1 


2 


3 


4 






AA 


All 




1 . Affected campus discussions of 
undergraduate education 

2. Contributed to faculty 




2.12 


2.28 




satisfaction 




1.71 


1.69 




3. Contributed to faculty interest 
in teaching 

4. Led to changes in instructional 




1.86 


1.88 




or teaching methods used 
5. Contributed to student 




2.41 


2.45 




satisfaction 




1.99 


2.03 




6. Affected student retention or 
graduation rates 

7. Affected student grade 




2.24 


2.20 




performance 

8. Affected student achievement on 




2.08 


1.95 




external examinations (e.g. 
professional licensure, GRE) 




2.01 


1.97 




External Impacts 




AA 


All 




9. Affected student applications or 










student acceptance rates 




1.40 


1.48 




10. Affected allocation or share of 










state funding 




1.55 


1.46 




1 1 . Affected evaluation from 
regional accreditation agency 

12. Affected private fund-raising 




2.47 


2.55 




results 




1.28 


1.42 




13. Affected success on grant 










applications 




1.69 


1.65 




14. Affected communication with 










external constituents 




1.65 


1.75 




15. Affected institutional reputation 










or image 




1.91 


1.94 





Note: * denotes difference greater than or equal to .3 or 10% from average of all institutions. 
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VI. Further Studies - Optional 



This page will be removed from the questionnaire before it is processed and completion of it is optional. However, 
we would like to know more about your institution’s experience with student assessment and we would like to be 
able to respond to you personally with a follow up report. 



Within the next year several institutions will be invited to participate in a more intensive study of the impacts of 
their student assessment practices and policies. Would you be interested in participating in a case study? 

yes 

possibly 

no 



If you are interested, we would appreciate any additional information regarding your student assessment practices that 
you believe would be of interest to other institutions. If you believe your approach to student assessment or its 
impacts are unusual, please describe it briefly (or enclose a report you think captures your experience). 



Please provide your name and address if you are interested in receiving a personal summary report of this survey. 

Name: : 

Title: 

Institution: 

Address: 



Phone: 

E-Mail: 



Thank you for taking the time to complete this instrument. 
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Appendix B 



Construction and 
Content of Variables 
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Variable 

Institutional Characteristics 



Definition 



Enrollment 


Single item. Reflects number of students enrolled in institution. Data from 
1995 Integrated Postsecondary Education Data System (IPEDS). 


Control 


Single item. (1 = public; 0 = private). Data from IPEDS. 


Institutional type 


Four dummy-coded single items. Reflects the institution’s Carnegie type. 
(Associate of Arts, Baccalaureate, Doctoral, and Research. Master’s institutions 
was the omitted category.) Data from IPEDS. 



External Influences on Student Assessment 

State initiative Single item. Reflects whether the state’s assessment initiatives were guided by 





legislative or other means (1 = no state plan; 2 = state policy; 3 = state statute; 4 
= combination of policy & statute). Data from SAS. 


State approach 


Single item. Reflects whether states mandate common indicators and outcomes 
(1 = no indicators or outcomes; 2 = institutional specific; 3 = common for some; 
4 = common for all). Data from SAS. 


Accrediting association 


Five dummy-coded single items. Reflects the institution’s regional accreditation 
association membership (Middle States; North Central; New England; Southern; 
Western. Northwest region was the omitted region). Data from IPEDS. 


Development of state plan 


Single item. (Ill A 1) Reflects how state plan for student assessment was 
primarily developed (1 = state; 2 = joint consultation between state and 
institution; 3 = no state plan or requirement). 


State influence 


Four single items. (Ill A 2 a-d) Reflect the influence of state requirements on the 
institutions assessment activities: a = important reason to initiate student 
assessment; b = increased institution’s involvement in assessment; c = have not 
been a factor in assessment activities; d = have been negative influence on 
assessment activities (1 = yes; 0 = no). 


State reporting requirements 


Four single items. (Ill A 3 a-d) Reflect the state’s reporting requirements: a = 
evidence that assessment plan is in place; b = measurement of state mandated 
indicators; c = use of institutionally devised indicators; d = evidence of 
institutional use of assessment information (1 = yes; 0 = no). 


State review methods 


Four single items. (ID A 4 a-d) Reflect the method used by state to review the 
institutions assessment activities: a = reviewed by state officials; b = reviewed 
using external reviewers; c = required institutional self-review; d = no review 
occurred ( 1 = yes; 0 = no). 


State review criteria 


Five single items. (Ill A 5 a-e) Reflect the processes included in the state review 
of the institutions assessment activities: a = review of institutions process itself; 
b = compare student performance record with past record; c = compare student 
performance record with peer institutions; d = compare student performance record 
with other in state; e = other (1 = yes; 0 = no). 



Variable 


Definition 


Accrediting influence 


Four single items. (Ill B 2 a-d) Reflect the influence of regional accreditation 
agency requirements on the institutions assessment activities: a = important 
reason to initiate student assessment; b = increased institution’s involvement in 
assessment; c = have not been a factor in assessment activities; d = have been 
negative influence on assessment activities (1 = yes; 0 = no). 


Accrediting reporting 
requirements 


Five single items. (Ill B 3 a-e) Reflect the regional accreditation agency 
reporting requirements: a = evidence that assessment plan is in place; b = 
intended uses of assessment information; c = results of assessment; d = evidence 
of actual institutional use of assessment information; e = unfamiliar with regional 
accreditation requirements (1 = yes; 0 = no). 


! 

External sources of support 


Five single items. (Ill C 1 a-e) Reflect the sources of support received to 
improve student assessment practices: a = FIPSE; b = other federal agencies; c = 
state incentive program; d = private foundation or corporate source; e = no known 
external grants (1 = yes; 0 = no). 


Use of external services 


Four single items. (Ill C 2 a-d) Reflect the use of services offered by each of the 
following type of postsecondary organization: a = professional associations; b = 
regional accrediting association; c = state-level agency; d = consortium of 
institutions. Respondents could choose from the following services offered by 
each organization: organization not used or not available; consultation services; 
assessment conferences; training workshops; publications or research reports (1 = 
used; 0 = not used). 



Institutional Approach to Student Assessment 

Academic intentions Single item. (I A 1) Reflects extent to which institutions collect data on current 





student’s academic intentions or expectations (1 = not collected; 2 = collected for 
some students; 3 = collected for many students; 4 = collected for all students). 


Basic college-readiness skills 


Single item. (I A 2) Reflects extent to which institutions collect data on current 
student’s college-readiness skills (1 = not collected; 2 = collected for some 
students; 3 = collected for many students; 4 = collected for all students). 


Cognitive assessment 


Four item factorially-derived scale. (I A 3-6) Reflects the extent to which 
institutions collect data on current students’ cognitive performance: competence 
in major field; general education competencies; higher-order cognitive skills; 
vocational or professional skills ( 1 = not collected; 2 = collected for some 
students; 3 = collected for many students; 4 = collected for all students). 
Cronbach alpha = .71. 


Affective assessment 


Three item factorially-derived scale. (I A 7-9) Reflects the extent to which 
institutions collect data on current students’ affective development and 
satisfaction: experiences and involvement with institution; satisfaction with 
institution; personal growth and affective development (1 = not collected; 2 = . 
collected for some students; 3 = collected for many students; 4 = collected for all 
students). Cronbach alpha = .68. 


Academic progress 


Single item. (I A 10) Reflects extent to which institutions collect data on 
current student’s academic progress (1 = not collected; 2 = collected for some 
students; 3 = collected for many students; 4 = collected for all students). 



Variable 


Definition 


Post-college assessment 


Three item fac tori ally -derived scale. (I A 1 1,12,14) Reflects the extent to which 
institutions collect data from former students: vocational or professional 
outcomes; further education; satisfaction and experiences with institution after 
leaving (1 = not collected; 2 = collected for some students; 3 = collected for many 
students; 4 = collected for all students). Cronbach alpha = .83. 


Civic/social roles 


Single item. (I A 13) Reflects extent to which institutions collect data on 
former student’s civic or social roles in the community (1 = not collected; 2 = 
collected for some students; 3 = collected for many students; 4 = collected for 
all students). 


Timing of data collection 


Nine item additive index. (I A 1-9) Reflects when institutions collect data (1 = 
not collected; 2 = collected at one point in time; 3 = collected at entry and while 
enrolled, or while enrolled and at exit; 4 = collected at entry and at exit; 5 = 
collected at entry, while enrolled, and at exit). 


Number of instruments 


Nine item additive index. (IB 1-9) Reflects student assessment instruments 
(institutionally developed, state provided, and commercially available) used by 
institution to collect ten types of assessment information: student plans or 
expectations; basic college-readiness skills; higher-order cognitive skills; general 
education competencies; competence in major; vocational or professional skills; 
personal growth and affective development; experiences or involvement with 
institution; satisfaction with institution (1 = instrument used; 0 = instrument not 
used). 


Student-centered methods 


Four item factorially-derived scale. (I C 1-4) Reflects the extent to which 
institutions use innovative or nontraditional assessment methods: performance in 
capstone courses; portfolios or comprehensive projects; observations of student 
performance; individual interviews or focus groups (1 = not used; 2 = used in 
some units; 3 = used in most units; 4 = used in all units). Cronbach alpha = .61 . 


External methods 


Two item factorially-derived scale. (I C 8-9) Reflects the extent to which 
institutions use assessment methods that data from external constituencies: 
employer interviews or focus groups; alumni interviews or focus groups (1 = not 
used; 2 = used in some units; 3 = used in most units; 4 = used in all units). 
Cronbach alpha = .63. 


Transcript analysis 


Single item. (I C 5) Reflects extent to which institutions use transcript analysis 
to collect student assessment information (1 = not used; 2 = used in some units; 

3 = used in most units; 4 = used in all units). 


External examination 


Single item. (I C 6) Reflects extent to which institutions use external 
examinations to collect student assessment information (1 = not used; 2 = used in 
some units; 3 = used in most units; 4 = used in all units). 


Interviews of withdrawing 
students 


Single item. (I C 7) Reflects extent to which institutions use interviews with " 
withdrawing students to collect student assessment information (1 = not used; 2 = 
used in some units; 3 = used in most units; 4 = used in all units). 



Variable 


Definition 


Student sub-populations 


Four single items. (I D 1-4) Reflect the use of different assessment methods for 
the following different student populations: a = adult students; b = part-time 
students; c = minority students; d = distance education students (1 = different 
method; 2 = same method). 


Number of studies 


Nine item additive index. (I E 1-9) Reflects the number of studies institutions 
conduct on the relationship between aspects of students’ institutional experiences 
and performance: course-taking patterns; exposure to different teaching methods; 
patterns of student- faculty interaction; extra-curricular activities; residence 
arrangements; financial aid and/or employment; admission standards or policies; 
academic advising patterns; classroom, library and/or computing resources (1 = 
conduct study ; 0 = do not conduct study). 


Number of reports 


Five item additive index. (I F 1-5) Reflects the levels of aggregation at which 
student assessment data are provided as reports: institution-wide; schools or 
colleges; academic programs or departments; special populations or subgroups of 
students; by course or groups of courses (1 = report provided; 0 = report not 
provided). 



Organizational and Administrative Support Patterns 

Mission emphasis Three item additive index. (II A 1 a-c) Reflects institutions’ mission statement 



Internal purposes 


emphasis on undergraduate education and its assessment: emphasizes excellence 
in undergraduate education; identifies educational outcomes intended for students; 
refers to student assessment as important activity (1 = yes; 0 = no). 

Four item factorially-derived score. (II B 3-6) Reflects the importance of internal 
institutional purposes for undertaking student assessment: guiding undergraduate 
academic program improvement; improving achievement of undergraduate 
students; improving faculty instructional performance; guiding resource allocation 
decisions (1 = no importance; 2 = minor importance; 3 = moderate importance; 4 
= very important). Cronbach alpha = .79. 


Accreditation purposes 


Single item. (II B 1) Reflects importance of preparing for institutional 
accreditation self-study as a purpose for undertaking student assessment (1 = no 
importance; 2 = minor importance; 3 = moderate importance; 4 = very 
important). 


State purposes 


Single item. (II B 2) Reflects importance of meeting state reporting 
requirements as a purpose for undertaking student assessment (1 = no importance; 
2 = minor importance; 3 = moderate importance; 4 = very important). 


Administrative and governance 
activities 


Seven item additive index. (II C 1-7) Reflects the number of administrative or 
governance activities used by institutions to promote student assessment: annual 
institution-wide assessment forums or seminars; rewards or incentives for 
administrators promoting use of assessment in unit; incentives for academic units 
to use assessment information; assessment workshops for administrators; board 
of trustees committee addresses assessment; faculty governance committee 
addresses assessment; student representation on assessment committees (1 = yes; 

0 = no). 



Variable 

Administrative and faculty 
support 



Type of plan or policy 



Formal centralized policy 



Institution-wide planning 
group 

Breadth of assessment planning 
group 



Responsibility for planning 
group 



Approval authority 



Operating responsibility 



0 




Definition 

Four item additive index. (II D 2-5) Reflects the degree to which chief executive 
officer, academic and student affairs administrators, and faculty support student 
assessment (1 = very unsupportive; 2 = somewhat unsupportive; 3 = neutral or 
unknown; 4 = somewhat supportive; 5 = very supportive). 

Seven single items. (II E 1 a-g) Reflects the institutions plan or policy for 
student assessment: a = formally adopted plan or policy requiring assessment 
activities for all academic units; b = formally adopted plan or policy requiring 
assessment activities for some academic units; c = formally adopted plan or 
policy requiring all academic units to develop their own assessment plan; d = 
formally adopted plan or policy stipulating institution-wide activities to be 
conducted by central committee, office, or officer; e = has no formal plan or 
policy but academic units are encouraged to conduct their own assessment 
activities; f = is currently developing plan or policy; g = does not have an 
assessment plan or policy (1 = yes; 0 = no). 

Single item. (II E 1 a) Reflects institution has formal institutional plan or 
policy requiring specified student assessment activities of all academic units or 
programs (1 = yes; 0 = no). 

Single item. (II E 2) Reflects institution has institution-wide group for student 
assessment planning and policy setting (1 = yes; 0 = no). 

Nine item additive index. (II E 3 a-i) Reflects the number of internal members 
included in the institution’s assessment planning group: chief executive officer; 
academic affairs administrator(s)/staff; student affairs administrator(s)/staff; 
institutional research administrator(s)/staff; academic review and evaluation 
administrator(s) /staff; student assessment administrator(s)/staff; faculty; students; 
other. 

Seven single items. (II E 4 a-g) Reflect the internal members who have 
executive responsibility for the institution- wide group responsible for planning or 
policy-setting for assessment: a = academic affairs administrator; b = student 
affairs administrator; c = institutional research officer; d = academic review and 
evaluation officer; e = student assessment officer, f = faculty member; g = other 
(1 = yes; 0 = no). 

Eleven single items. (II E 5 a-k) Reflect the internal members who approve any 
changes to institutions assessment plan or policy: a = board of trustees; b = chief 
executive officer; c = chief academic affairs officer; d = chief student affairs officer; 
e = institutional research officer; f = academic review and evaluation officer; g = 
student assessment officer; h = student government; i = academic senate or other 
faculty committees; j = faculty union; k = other (1 = yes; 0 = no). 

Eight single items. (II E 6 a-h) Reflect the internal members who have 
operational responsibility for the institution’s day-to-day assessment activities: a 
= academic affairs administrator; b = student affairs administrator; c = institutional 
research officer; d = academic review and evaluation officer; e = student 
assessment officer; f = faculty member; g = other; h = no one (1 = yes; 0 = no). 
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Variable 


Definition 


Reporting relationship 


Six single items. (II E 7 a-f) Reflect the individual to whom person with day-to- 
day responsibility reports: a = chief executive officer; b = chief academic affairs 
officer; c = chief student affairs officer; d = institutional research officer; e = 
academic review and evaluation officer; f = other (1 = yes; 0 = no). 


Conducted evaluation 


Single item. (II F 1 a-d) Reflects if institution has formally or informally 
evaluated its student assessment process (1 = yes; 0 = no). 


Evaluations elements 


Eight single items. (II F 2 a-h) Reflect the elements that were reviewed during 
the institutions assessment evaluation: a = student assessment plan or policies; b 
= structure and responsibility for assessment; c = achievement of intended 
objectives; d = reliability and validity of instruments and methods; e = quality of 
data analysis; f = use of information in institutional decision-making; g = 
problems encountered; h = comparison of costs and benefits (1 = yes; 0 = no). 



Assessment Management Policies and Practices 

Budget decisions Two item additive index. (IV A 3-4) ; Reflects formal use of assessment 





information in the budget process: to competitively allocate resources among 
academic units; to reward academic units for improvement (1 = yes; 0 = no). 


Computer support 


Three item additive index. (IV B 2-4) Reflects institutional capacity to collect 
and manage student assessment information: computerized student information 
system includes student performance indicators; student information system tracks 
individual students; student assessment database integrated with other institutional 
databases (1 = yes; 0 = no). 


Access to information 


Five item additive index. (IV C 1-5) Reflects internal accessibility of 
assessment information on individual students by: institutional research or 
assessment professionals; senior academic administrators; department chairs or 
academic program administrators; student affairs professionals; faculty advisors (1 
= yes; 0 = no). 


Distribution of reports 


Six item additive index. (IV D 1-6) Reflects the number of constituent groups 
to whom student assessment reports are regularly distributed: students; faculty; 
academic administrators; student affairs professionals; employers; general public 
(1 = yes; 0 = no). 


Student involvement 


Three item factorially-derived scale. (IV E 1,3,4) Reflects the extent to which 
institutions have policies or practices to promote student involvement in 
assessment activities: inform students about assessment purposes and uses; 
require students to participate in assessment activities; provide students with 
individual feedback on assessment results (1 = not done at all; 2 = done in a few 
departments; 3 = done in some departments; 4 = done in many departments; 5 = 
done in most departments). Cronbach alpha = .69. 



Variable 

Professional development 



Student affairs training 



Faculty evaluation 



Academic planning and review 



Institutional Uses of Student 
Educational decisions 



Definition 

Four item factorially-derived scale. (IV F 2-5) Reflects existence of professional 
development policies or practices on student assessment for faculty and academic 
administrators: provide funds for faculty to attend or present at assessment 
conferences; offer student assessment workshops or consultation for faculty; 
provide assistance (e.g., paid leaves, stipends, course reduction) to improve 
faculty use of student assessment; provide student assessment workshops for 
academic administrators (1 = not done at all; 2 = done in a few departments; 3 = 
done in some departments; 4 = done in many departments; 5 = done in most 
departments). Cronbach alpha = .77. 

Two item factorially-derived scale. (IV F 6-7) Reflects existence of professional 
development policies or practices on student assessment for student affairs 
personnel: require assessment training for student affairs staff; provide student 
assessment workshops for student affairs administrators (1 = not done at all; 2 = 
done in a few departments; 3 = done in some departments; 4 = done in many 
departments; 5 = done in most departments). Cronbach alpha = .84. 

Five item factorially-derived scale. (IV G lf5). Reflects existence of faculty 
evaluation and reward policies and practices related to student assessment: 
promotion evaluation considers evidence of student performance; salary evaluation 
considers evidence of student performance; promotion, tenure or salary reviews 
consider faculty participation in student assessment; promotion, tenure or salary 
reviews consider scholarship on assessment; public recognition or awards for 
faculty use of student assessment (1 = not done at all; 2 = done in a few 
departments; 3 = done in some departments; 4 = done in many departments; 5 = 
done in most departments). Cronbach alpha = .77. 

Four item factorially-derived scale. (IV H 1-4) Reflects the incorporation of 
student assessment data into academic planning and review processes for: 
academic departments or undergraduate programs; general education or core 
curriculum; courses; student academic support services (1 = not done at all; 2 = 
done in a few departments; 3 = done in some depsulments; 4 = done in many 
departments; 5 = done in most departments). Cronbach alpha = .84. 

Assessment 

Ten item factorially-derived scale/ (V A- 1-5, 8-12) Reflects the influence of 
student assessment information in educational decisions: revision of 
undergraduate academic mission or goals; designing or reorganizing academic 
programs or majors; designing or reorganizing student affairs units; allocating 
resources to academic units; modifying student assessment plans, policies, or 
processes; revising or modifying general education curriculum; creating or 
modifying student out-of-class learning experiences; creating or modifying 
distance learning initiatives; modifying instructional or teaching methods; 
modifying student academic support services (1 = no action or influence known; 

_2 = action taken, data not influential; 3 ^ action taken, data somewhat influential; 
4 = action taken, data very influential), Cronbach alpha = .83. 
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Variable 

Faculty decisions 



Definition 



Two item factorially-derived scale. (V A 6-7) Reflects the influence of student 
assessment information in faculty decisions: deciding faculty promotion and 
tenure; deciding faculty salary increases or rewards (1 = no action or influence 
known; 2 = action taken, data not influential; 3 = action taken, data somewhat 
influential; 4 = action taken, data very influential). Cronbach alpha = .79. 

Institutional Impacts of Student Assessment 

Faculty impacts Four item factorially-derived scale. (V B 1-4) Reflects student assessment 

impacts on faculty: affected campus discussions of undergraduate education; 
contributed to faculty satisfaction; contributed to faculty interest in teaching; led 
to changes in teaching methods used (1 = not monitored, do not know; 2 = 
monitored, negative impact; 3 = monitored, no known impact; 4 = monitored, 
positive impact). Cronbach alpha = .79. 

Four item factorially-derived scale. (V B 5-8) Reflects student assessment 
impacts on students: contributed to student satisfaction; affected student retention 
or graduation rates; affected student grade performance; affected student 
achievement on external examinations (1 = not monitored, do not know; 2 = 
monitored, negative impact; 3 = monitored, no known impact; 4 = monitored, 
positive impact). Cronbach alpha = .82. . 

Seven item factorially-derived scale. (V B 9-15) Reflects student assessment 
impacts on external constituents: affected student applications or acceptance rates; 
affected allocation or share of state funding; affected evaluation from regional 
accrediting agency; affected private fund-raising results; affected success on grant 
applications; affected communications with external constituents; affected 
institutional reputation or image (1 = not monitored, do not know; 2 = 
monitored, negative impact; 3 = monitored, no known impact; 4 = monitored, 
positive impact). Cronbach alpha = .82. 

Assessment of Teachin^and Learning for Improvement and Public Accountability: State Governing. Coordinating 
Board and Regional Accreditation Association Policies and Practices (Cole et al„ 1997) 



Student impacts 



•External impacts 
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